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David H. Jonassen, University of Colorado at Denver 274 

A NEW COMPUTER-BASED LEARNING EVNIRONMENT THE HYPERMEDIA-ASSISTED KNOWLEDGE 
CONSTRUCTION SYSTEM 

William Lidwell, Thomas Troutman, and David B.Palumbo , 

University of Houston, Clear Lake 279 

HYPERTEXT DELP/ERY SYSTEMS 

Carol B. MacKnight. University of Massachusetts 

Santosh Balagopalan, Mentor Graphics 

Special Interest Group for Interactive Video-Audio (SIGIVA) 

DIGITAL AUDIO IN MULTIMEDIA INSTRUCTION 

Ann Barron, University of South Florida 295 

HIGHLY INTERACTIVE PROGRAMS WITH VIDEO FOR LEARNING LANGUAGES 
Rika Yoshii, Ph.D., Alastair Milne and Alfred Bork, Ph.D. 

University of California 300 

ACCESSING DATABASES FROM AN IBM INFOWINDOW COMPUTER-BASED VIDEO INSTRUCTION 

PROGRAM USING "C" LANGUAGE PROCEDURES 

C. Michael Hassett and Elizabeth A. Unger, Kansas State University 

Mary R Hassett, Fort Hays State University. 308 

COMFUTER-ASSISTED LANGUAGE LEARNING (CALL) THROUGH THE LENS OF ELABORATION 
THEORY 

Anita M. Knisbacher, U.S. Government Department of Defense «* 17 

DEVELOPING AN INTERACTIVE MULTIMEDIA NETWORK FOR DISTANCE EDUCATION 
Dent M. Rhodes and David Doss, 

Illinois State University 328 

MICAL AN INTERACTIVE MULTIMEDIA RESEARCH PROJECT IN COMPUTER SCIENCE 
Dr. D. V. Pigford and Dr. Greg Baur 

Western Kentucky University 333 

Special Interest Group for PILOT (SIGPILOT) 
AUTHORING SOFTWARE SELECTION 

Michael Collins, Memorial University of Newfoundland 338 

DEVELOPING CBT COURSES FOR END-USER TRAINING 

Douglas MacKenzie, DMC Ltd 345 

Special Interest Group for PLATO Users' Group (PUG) 

I NEED SOME HELP 

Leslie Smith 353 
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Special Interest Group for Theory and Research (SIGTAR) 

WORKING ALONE VS. WORKING IN TEAMS DURING COMPUTER BASED INSTRUCTION IN PUBLIC 
SPEAKING 

Michael R Hemphill Ph.D.. Christina C. Standerfer, University of Arkansas at Little Rock 356 

GENERATIVE PROCESSING OF STRUCTURAL KNOWLEDGE IN HYPERTEXT 

David H. Jonassen, University of Colorado 360 

EXTERNAL EVALUATION OF POSIT 

Michael A, Orey, The University of Georgia 365 

KNOWLEDGE ACQUISITION AND KNOWLEDGE REPRESENTATION IN INTELLIGENT COMPUTER- 
AIDED 

INSTRUCTION 

Joo-sung Park,and Michael Orey, 

The University of Georgia 374 

INTEGRATING COGNITIVE THEORY INTO GAGNE'S INSTRUCTIONAL EVENTS 
Michael A. Orey, James R. Okey, Marshall G. Jones and Linda S. Stanley 

The University of Georgia 377 

LEARNER CONTROL AND MOTIVATION IN COMPUTER-BASED INSTRUCTION 
Joo-sung Park, Thomas C. Reeves, Ph.D. 

The University of Georgia 333 

TECHNOLOGY USING TEACHERS ARE THEY A DIFFERENT TYPE? 
Carolyn Rude-Parkins, Joseph M. Petrosko, Jr. University of Louisville 

Ivan Baugh, Jefferson County Public Schools 394 

COMPUTER-BASED COGNITIVE APPRENTICESHIPS AN EXAMPLE FROM WEATHER FORECASTING 
Brent Wilson, University of Colorado 
Brian Heckman, UCAR/COMET 

Sherwood Wang, University of Colorado 402 

Special Interest Group for Telecommunications (TELESIG) 

AMODEL FOR THE USE OF TELECOMMUNICATIONS TECHNOLOGY AND MULTI-MEDIA SYSTEMS 

CREATE REAL-TIME MULTIPLE-USER COMPUTER-BASED LEARNI NG ENVIRONMENTS 

Dr. Greg Baur Western Kentucky University 408 

PREPNET CONNECTING PENNSYLVANIA AND BEYOND 
Blair R. Bernhardt, and Denlse LePag 

Lehigh University 412 

SPEAKERS ADDRESSES 416 
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INIGO, A NEW AUTHORING SYSTEM 



Gerdy C. ten Bruggencate , University ofTwente 
Eelco F. Laagland, University ofTwente 

The Increased acceptance of the Microsoft Windows graphical user Interface calls for the development 
of computer assisted learning (CAL) material that will run In this environment. The Inlgo authoring 
system Is developed to facilitate the production of courseware that will run with Windows. 

Inlgo enables the courseware developer to create CAL material (courseware) that Is student controlled. 
The courseware, as a Windows application, runs In a window that can be arranged on the screen and 
resized. Within the CAL window the student can handle several windows (Including windows 
containing video images), choose menu options, press command buttons, etc., to create his own 
learning environment and control the learning process. When another Windows application, for 
example a spreadsheet or a calculator, runs at the same time, exchange of Information between the 
courseware and the other application Is possible. The exchange of information can be student driven 
by means of the clipboard, or program driven by means of Dynamic Data Exchange (DDE). Inlgo 
contains a DDE protocol enabling other Windows applications to control Inlgo and request 
information (course data) from the currently running course. Furthermore, the data exchange features 
offer the opportunity to develop In Inlgo an educational environment for e.g. a simulation package. 

To explore the possibilities of Inlgo and in order to Improve the system and to adapt it to the needs of 
courseware developers a first courseware development project has been started In January 1990. The 
courseware on fundamentals of digital design will be ready in September 1991. 



CONNECTING PEOPLE AND TECHNOLOGY THROUGH 
OPEN-ACCESS COMPUTING: FACILITY DESIGN FACTORS 



Jeanette S. Gates, PttD., Instructional Development 

As resources tighten, open-access computing facilities are often seen as a means to use resources more 
efficiently. Open-access refers to facilities that are available to all students and faculty, regardless of 
discipline. Within a single area, open-access users may work on assignments In history or nutrition, 
review their algebra skills, or prepare a paper for English. 

Because of the variety cf activities being conducted In a single area, open-access facilities should not 
be designed or operated In the same manner as traditional ' computer labs". For example, desk space 
may need to be wider and lighting brighter when students are taking notes or working from a hand- 
written draft, as opposed to composing on the screen. Yet, most designers-be they architects or college 
administrators- -are unaware of the differences In the design considerations that go into an open- 
access area. At the same time, the literature provides very little guidance, particularly in the level of 
detail required for actual facility design. 

Over the past eight years, the author has designed and opened eight open-access computing facilities. 
Lessons learned from operating a facility after designing it have been incorporated into the next 
design. A variety of tools, as well as specifications for the electrical, space, furniture, and 
environmental concerns have been documented. This presentation will share the knowledge 
accumulated through this experience. 




CONNECTIONS TO RESOURCES FOR INSTRUCTIONAL 
COMPUTING IN HIGHER EDUCATION 



Carol A. Dwyer, Perm State University 

Many higher education institutions have a support unit which has as part of its mission to help 
faculty use interactive computer technologies for teaching and learning. These units often are small 
but, nevertheless, have more requests for assistance than they can handle. 

CBEL—Teaching and Learning Technologies Gro up at Penn State has been providing support services 
to faculty for several years. During that time wt have become aware of a number of resources which 
assist in our work in empowering faculty to integrate computer technologies and educational 
applications into their curriculum. 

The database discussed in this presentation is a means by which we can share some of the information 
we have gathered. The database includes: 

1. sources of high-quality and award-winning software, 

2. organizations which focus on uses of technology in education. 

3. publications focusing on instructional applications in higher education, 

4. conferences on the use of technology in education and training. 

5. videotapes showcasing software and interactive video applications, and 

6. books on Instructional design 

Database fields include a brief description of the resource, address of the company or organization 
which will provide further information, phone, electronic address, if available, and cost, if available. 

Participants may have a printout of the database during the session. A disk containing the database 
in Macintosh format may be requested. 
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AN OPEN-STRUCTURED INTERACTIVE TEST 
AND EXERCISE MANAGEMENT SYSTEM 



Pieter Jansen, Deljl University of Technology 
Eugane v.d, Kamp, Deljl University of Technology 
Roy Natadarma. Delft University oj Technology 

One of the educational measures to motivate students or to assess their achievement is to have 
students solve questions and give feedback on the results. Itembanking may be considered to support 
structured access to items concerning a particular subject, a specific educational objective or a specific 
question type. 

In this presentation a description is given of an open structured Interactive Test and Exercise 
Management System. ITEMS-D (-Delft). The tool has been developed to support three types of 
educational activities: 

1. Fully interactive tests, exercises or examinations; 

2. Paper and pencil exercises and Interactive sessions to enter the answers; 

3. Paper and pencil examinations with optical mark forms. 

Aspects of ITEMS-D to be described: 

- Functions to support user-defined item development and item runtime environments. 

- Commercially available wordprocessors or authoring tools may be Integrated into the system to 
develop paper and pencil questions or interactive questions; 

- Functions to support the use of variable parameters in questions; 

- (replaceable) formula-evaluator; 

- The use of profiles to describe student applications based on the itembank; 

- Composing the (menu-)interface to direct students to the appropriate application and to run the 
different item environments using a LAN-network; 

- Functions to assess and manage student results and to perform test analysis and item analysis. 

Results of pilot projects at the Delft University of Technology will be presented. 
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THE GOLDEN ARCHES OP ACADEMIC COMPUTING 



Dr. Kenneth L. Modesitt, Western Kentucky University 

"Press On: Nothing in the world can take the place of 
PERSISTENCE. 

0 Talent will not -- Nothing Is more common than unsuccessful men 
Mc) with talent. 

0 Genius will not -- Unrewarded genius is almost a proverb. 
0 Education alone will not — The world is full of educated derelicts. 
Persistence and determination alone are omnipotent." 
From a McDonald's Ad, 1989 

In a paper presented by the author at ADCIS in 1989 (Modesitt, 1989), he spoke of the many lessons 
learned about computer based learning, based n spending three decades in the field. In this 
presentation, an update is given, showing that "persistence" does pay off. This determination has 
resulted in the campus of a comprehensive university of 15,' XX) students having, for the first time, all 
of the resources necessary to make widespread use of computing In instruction more than Just a 
fantasy. 

The recommendations given in the 1989 paper were: 

fashion, use software engineering and expert system principles, 
contribute to new areas, also involving students, and provide 
organizational-wide resources. 

The planning done at that time included several components: hardware, communication, computer- 
based learning, organizations, and people. This presentation will discuss each of these in turn, 
indicating past (circa 1987), present (1991), and desired future (circa 19 r ") for each. The 
recommendations for the next period of growth will spread out the esources across the Commonwealth 
of Kentucky to K-12 schools, government, usiness, and industry. 

1. Modesitt, K. "Lessons Learned in Computer-based Learning: A Personal Tale of Three Decade j." 
Association for the Development of Computer-based Instructional Systems (ADCIS) Conference, 
Washington, D.C., 1989, pp. 174-180. 
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A DEPARTMENT GETS A LAN, OR "YOU'RE NOT 
GOING TO DO THAT TO MY COMPUTER!" 



Donald Scherer, Bowling Green State University 

This paper reports the financial, political and educational complexities of installing an AppleTalk 
network in an academic department. 

The financial problem arises primarily from the conflict between the University's policy of servicing 
only IBM and Apple machines and the less expensive IBM clones the department can buy. 

Political problems attend the contrast between administrative and faculty LAN usefulness, the 
traditional prerogatives of secretaries, the diversity of faculty uses for a LAN and the security 
problems of integrating graduate students into the LAN. Problems surrounding the integrating of 
graduate students arise from both the transience of graduate students and their dual roles as students 
and instructors. 

The department has also needed to educate itself about which of its many computer uses will be 
networked. In addition to word processing peripherals on the server, communicating registration 
information and requisitions through the University's mainframe and access to bulletin boards, E- 
mail, computerized information retrieval, "past master" text accessibility and complete philosophic 
bibliographic information on CD ROM , different philosophers use computers differently: MIDI for 
aestheticians, modelling for philosophers of mind and simulations for the environmental ethicist. 



INTEGRATED LEARNING SYSTEMS AND ADULT BASIC SKILLS 



Robin Taylor, PhD, New Century Education Corporation 

Widespread efforts are being undertaken - by public educational institutions, by volunteers, and by 
business and industry - to remedy the growing problem of illiteracy and innumeracy in the United 
States. The President's agenda of educational initiatives has heralded the birth of still more projects 
focused, in part, on this pervasive problem. 

A great variety of instructional methods (classroom, tutoring, self-study, CAI) have been employed in 
our efforts to improve adult basic skills, with varying degrees of effectiveness. One of the more recent 
approaches to be tried is the integrated leamJDg system (ILS). 

Integrated learning systems are proving to be particularly effective in attacking the extensive problem 
of adult basic skills. An ILS can serve many students simultaneously, is adaptive to individual needs, 
and provides continuous monitoring and reporting of student progress. ILSs are being implemented 
successfully in sites as diverse as community colleges, government-funded adult education centers, 
corporate training facilities, and labor union-sponsored training facilities. 

The Hudson Institute, in its report, Workforce 2000. has predicted that the magnitude of the adult- 
iUiteracy problem will increase throughout this decade and into the next century, as workforce skills 
lag far behind workplace needs. Will ILSs continue to be a viable solution? 



RESEARCH PROJECT "INTERACTIVE TECHNOLOGY 
IN HIGHER TECHNICAL EDUCATION" 



Jarmo Viteli, University of Tampere, Finland 

There is a growing demand in Finnish Technical education to develop it toward the idea of an open 
learning environment. One part of this process is to help teachers to utilize interactive technology in 
their work. The research project "Interactive Technology in Higher Technical Education" is part of 
developing project 'Teacher Education For Higher Technical Education". In Finland there is an 
obligatory teacher education for those whose aim is to become a teacher to Technical College's. 

The research project started in fall 1990. The first question to answer was: How many lecturers in 
technical education were using interactive technology in their courses? Surprisingly, (or not! the use 
of interactive technology in technical education showed to be very low. Second question: Why don't 
you use interactive technology in your courses? The answers varied a lot but one of the main 
explanation or excuse was the lack of qualitative software. 

After that we analyzed existing educational software for technical education in Finland. We found 
almost one hundred CAI-programs which were evaluated by lectures from technical colleges. The 
preliminary results will report in this presentation. 

We send an open "Call for ICE-BREAKERS" to all technical colleges in Finland to invite lectures who 
would like put their time to develop instruction by using interactive technology. There were a lot of 
interest for this and this presentation will report the first steps of this groups ideas, training, results 
and problems. 
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ASSESSMENT OP COMPUTER-BASED INSTRUCTION (CBI) IN 
CRIMINAL JUSTICE EDUCATION AND TRAINING PROGRAMS 



James B. Wells, North Georgia College 
Ben Layne, Georgia State University 
Claudia Flowers, Georgia State University 

Although some published studies have been performed with computer-based instruction (CBI) in 
criminal Justice environments, little knowledge of the extent to which CBI is used in criminal Justice 
education anu training exists. The purpose of this study was to investigate the extent to which CBI is 
being used nationally in criminal Justice education and training programs. The target population 
consisted of all departmental chairmen, training directors, and agency heads in the United States 
who coordinate or supervise adult criminal Justice education or training programs in training 
centers, technical institutes, academies, community colleges, undergraduate and graduate colleges 
and universities. The results from a self-administered mail survey (copy provided) utilizing a 
sampling frame consisting of a directory of criminal Justice educators, assessed the extent to which 
CBI is being used. Key areas of interest included identifying the types of education and training 
programs that are using CBI, as well as those that are not. Reasons for CBI usage as well as non-usage 
were also investigated. The results of this survey present concerned criminal Justice educators an 
opportunity to examine computer-based instruction applications in criminal Justice education and 
training settings, as well as indicate a need by some programs to expand their use of computers into a 
specific teaching area. Applicability of this survey toward assessing CBI usage in other disciplines is 
also discussed. 
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ERGONOMIC CBT 



Kay Bonham, Goal Systems International 
Linda Randolph Goal Systems International 

Ergonomics is a design philosophy that considers various aspects of usability. For example, 
efficiency, ease of use, and comfort level are central ergonomic principles. This session describes 
ergonomics and explains how sound ergonomic design principles can be applied specifically to the 
design and development of quality computer-based training courses. 

This session will analyze the usability of everyday objects to illustrate the importance of applying 
ergonomic principles during the design process. It will describe a systematic ergonomic design 
procedure and will compare this procedure to the CBT development process. 

This session will also discuss the benefits of using ergonomic design principles in CBT development, 
as well as the problems that arise from not following ergonomic principles. It will encourage 
participants to apply an ergonomic approach to various phases of the CBT design process, from how to 
accommodate diverse learning styles to the best way to present information on the screen. 



COMPUTER-BASED TRAINING MODEL FOR A 
GEOGRAPHIC INFORMATION SYSTEM 



Debra A. Brtnegar, U.S. Army Construction Engineering Research Laboratory 

A Geographic Information System (GIS) is a complex computer system used In solving difficult 
planning and management problems related to land, facilities and the environment. In the last few 
years, the GIS user community has gone from consisting of only computer programmers and GIS 
experts to the land manager's assistant and college students. This movement from an expert-to- 
novice user community has brought with it Inherent training problems. The complexity of the system 
may produce these problems yet, In turn, offers an advanced technological environment in which to 
train and support users. Most GISs use state-of-the-art workstations which provide access to multi- 
media (i.e., graphics, text, video) simultaneously. 

This session discusses current research being performed in the area of GIS training that looks into 
how to best design a total performance support system in this type of environment. A training model 
will be presented which provides instructional designers with an insight Into issues that need to be 
addressed when designing a support environment for users of GISs and other similar systems. 
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AUTOMATED INSTRUCTIONAL DESIGN AT MEDIASHARE INC. 



Ann Marie Canjteld, Utah State University 

Utah State University and MediaShare, Inc. have teamed up to design and develop a performance 
support system for salespeople. The project is in its first year of development and prototypes of 
version 1.0 of the system will be demonstrated at this showcase. The system is designed to deliver 
product training for salespeople and service departments as well as provide Interactive presentations 
for customers. The system contains a knowledge Indexing framework which enables a hypermedia 
system to contain meaningful links and avoid the problems of losing users in hyperwilderness. The 
system is based on the Second Generation Instructional Design theories also In development at Utah 
State University under the direction of M. David Merrill. 



OUTGROWING THE COTTAGE: DEVELOPING 
COMPUTER-BASED TRAINING ON A LARGE SCALE 



Robert C. FrattnU AT&T National Product Training Center 

Developing, maintaining, and supporting two computer-based training courses is more than twice as 
difficult as doing the same for a single CBT product. Managing more and more CBT deliverables 
requires more than simply an Increase In the scale of operations. There is a qualitative as well as a 
quantitative change In the nature of the tasks required. 

The Electronic Training Solutions (ETS) department of the AT&T Product Training Center has 
learned through the experience of such growing pains. By 1991, the department was responsible for 
the development, maintenance, and support of students using several hundred electronic training 
deliverables worldwide running on a range of delivery platforms. 

The department had recognized beginning in 1989 the fact that its organizational mission and role 
was changing. Internal processes and software tools were analyzed in terms of their ability to meet 
the needs of courseware production on the current scale, as well as organizational skills and the 
corporate product management environment. 

This presentation discusses the results of our analysis and provides a status report of our efforts. 
Software tools, development processes, and the role of CBT specialists have all changed in this new 
courseware management process. The techniques that we have used are appropriate for any electronic 
training development group which seeks to tailor their processes and tools to their specific 
organizational environment. 
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COCKPIT RESOT r ACE MANAGEMENT TRAINING: 
MULTIMEDIA SITUATED LEARNING 



Hilary McLellan, Kansas State University 

Line-Oriented Flight Training (LOFT) for flight crews, developed to promote effective cockpit resource 
management, is an example of situated learning in a sophisticated multimedia training 
environment. All six of the critical situated learning components — Apprenticeship; Collaboration; 
5! iJS!2S : Coachln & Multiple practice; Articulation — are present in the LOFT training program. 
The LOFT program has several important implications for designers seeking to implement training in 
accordance with the Situated Learning Model: 1) the debriefing or reflection process for a team of 
learners, rather than an individual learner must be designed to provide feedback for both the team 
and the individual: 2) it Is essential to differentiate between training and testing in the situated 
earning environment and to make sure that this distinction is clear to both the Instructor and the 
learner; 3) the role of the Instructor in a situated learning environment must be carefully designed as 
part of the overall design of the situated learning training program; 4) the selection and training of 
the instructor must be explicitly considered and planned for In situated learning, as in other training 
contexts; 5) the pacing and coordination of complex interrelated events in the situated learning 
scenario or script must be carefully designed In order to promote both realism and learning; and 6) it 
is necessary to design customized scenarios to meet the needs of different learners 



THE REMOTE MECHANICAL "C" LINE COURSE: 
A DESKTOP SIMULATION SOLUTION 



M.M. Prather, Westinghouse Hanford Company 
K.R. Nuttall, Boise Cascade Corporation 

Westinghouse Hanford Company has developed a desktop simulation-based training program for the 
operation of the Remote Mechanical "C" Line process in the Plutonium Flashing Plant at the 
Hanford Site, Richland, Washington. Simulations display and continually update values of svstem 
parameters on computer graphics of Remote Mechanical "C" Line equipment. Students use a varietv of 
controllers to maintain proper system status. Instructors may select faults to be introduced Into some 
simulations to test student responses to off-normal events. Simulations are preceded by computer- 
2r!S iw < f* U 2 ' uncti0I V Processes, operation, and error conditions associated with each part 
of the line. The tutorials are also interactive; they include opportunities to use each of the controls- 
valves heaters, agitators, etc.. used to operate the individual processes. Phase I of the course was 
completed in December 1990. and the complete Geneial Concepts course was delivered in July 1991 
Instructors report that student response to the course has been extremely favorable. 

fi^ 111 2ft perspective - Remote Mechanical "C" Line course represents one step in the diffusion of 
the well-known and well-documented simulator training activities for nuclear reactor operators to 
Mi^ Q J!^^. P , ^0g^amS • Because °f the lengthy response times of the actual process, the Remote 
Mechanical C Line course can retain many of the capabilities of practice and testing in a simulated 
work environment while avoiding the cost of a full scale simulator and the exposure and waste 
developed by practice runs of the Remote Mechanical "C" Line itself. From another perspective the 
Remote Mechanical C Line course suggests training advances even beyond the most faithful 
simulators. For example, by combining computer-based training lessons with simulations this 
course permits students to focus on specific processes occurring inside chosen components In effect 
an interactive training manual is available on line with the simulation itself romponems - In ellect ' 
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DELIVERING CBT TO YOUR USERS: DESIGN AND 
IMPLEMENTATION ISSUES FOR SUCCESSFUL LEARNING 



Adele Suchtnsky, U.S. General Accounting Office 
Catherine C. McMahon, WORDPRO, Inc. 

A wealth of excellent off the shelf Computer Based Training (CBT) is available for use by business, 
government and academia. Top management support, however, is not necessarily the only element 
required to ensure the successful implementation of a multi-media Learning Center or a distributed 
training program. A well defined understanding of the Job functions performed by the user 
population, clearly defined training/learning objectives, a well designed and equipped facility, 
appropriate staffing, marketing the new programs, effective procedures, and, above all, the 
appropriate up front and on-going selection of courseware (CBT, IVD), etc to address the target 
population is critical to successful Implementation. 

The initial challenge is to identify the place of the Learning Center within the context of the training 
environment. The key issues include methodologies for courseware selection, gaining organizational 
support for the program, determining the best ways to get the training to the users, determining 
when/how self-paced training can enhance and/or support the existing instructor-led training 
program, and determining how to maintain the viability and acceptance of the program In the future. 

This presentation will discuss how to's and lessons learned in one successful case study. It Is designed 
for the training professional charged with the responsibility to implement CBT in a Learning Center 
environment or distributed training program who would like to improve his/her current operation. 



WHAT DO WE REALLY KNOW ABOUT SMALL GROUP CBT 



Theodore M. Shlechter, Army Research Institute 

Foremost among many recent trends in education has been a surge of interest in small group learning. 
A search of ERIC and DITIC have produced nearly 50 works since 1985 on small group CBT. A number 
of claims have been made for small group CBT. It has been claimed for example, that small group CBT 
will help students elaborate upon the instructional materials and thus increase their llkellihood of 
retaining the information. Unfortunately, most of these studies have focused on examining strategies 
for making small group CBT more instructionally effective while ignoring the more basic issue-the 
relative instructional value of small group versus individualized CBT. Those few studies, which have 
actually examined this issue, have provided inconsistent findings. Shlechter (1990), for example, has 
found that small group CBT led to more lasting retention of procedural instructional materials than 
did individualized CBT; while this effect was not found by others (e.g.. Daltor ,1991: Carrier & Sayles, 
1987). A consistent finding in this literature is that students in individual ^d CBT situations very 
rarely outperformed those in small group situations. Indeed, small group k.. ming may make CBT 
more instructionally efficient with more students using fewer computers without harming the 
instructional process. 
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THE MANAGEMENT CHALLENGE-CBT AT THE 
COMMONWEALTH BANK OP AUSTRALIA 



R J Spence, Commonwealth Bank of Australia 

The Commonwealth Bank of Australia is represented nationally across a network of 1500 branch 
offices. Training needs, coupled with distance education challenges, has seen the Installation of 1200 
stand-alone PCs dedicated to CBT. This session outlines the processes that were involved in 
establishing this large-scale CBT system and then concentrates on system acceptance, maintenance 
and the management of a courseware development group of 25. Present day challenges include a 
change In the delivery platform operating system to OS/2. 



USING A WORD PROCESSOR TO DEVELOP 
COMPUTER-BASED INSTRUCTION 



Kevin Stall, Systems , Requirements , and Services Associates ,Inc. 

Many developers and designers neglect one of the most obvious tools that make a programmers life 
easier. Using a word processor can increase not only the developers and designers productivity, but 
also the programmers. By using macros and other techniques the programmer will be able to do all 
the programming utilizing the word processor. This is particularly true with PLATO. 



OBJECT-ORIENTED PROGRAMMING: A PASSING PAD 
OR THE COMMING OF A NEW GENERATION 



Jennifer A. Chen-Troestcr, Utah State University 
Zhongmin Li University of Utah 

Object-oriented programming (OOP) is becoming the programming methodology of the 1990's. 
Through the use of an object model, encapsulation, abstraction and inheritance, object-oriented 
programming enables programmers to easily reuse, extend and modify code. We will discuss the basic 
elements of OOP, how object-oriented design is different from traditional procedural software design 
and also look at designing a project using two object-oriented tools, HyperCard and ToolBook. 
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COMPUTER-BASED INSTRUCTION (CBQ MODELS 



Peter J. Rlzza Jr., Princeton Center for Education Services, Inc.; 
Votkert Balk, University of Amsterdam; and 
Franklin C. Roberts, Western Washington Univ. 

Courseware is the term used to refer to the range of possible computer based instructional (CBI) 
lessons. Courseware is used to deliver instruction on virtually all topics, employing a variety of 
strategies and approaches. This variety is one of the characteristics of CBI design that makes it an 
extremely labor Intensive task. 

While courseware is often varied in both its purpose and format, there is a common set of strategies 
and procedures that are frequently used in the design of CBI materials. There are literally dozens of 
approaches that can aid the designer In creating effective lessons. 

By systematically applying these basic strategies, it is possible to reduce the amount of time required 
to create lessons, as well as ensure a level of consistency and instructional integrity in what is 
produced. At the same time, they help to focus the CBI development process on the more important 
and creative aspects of instructional design. 

This session will outline the rationale for a set of models which can be used in the courseware design 
process. These models can be used to create a lessons that are complete, thorough, instructionally 
sound, and have a consistent interface. 

This session will present the standard navigation and record keeping architecture for the router 
which supports the CBI Models and launches other "off-the-shelf ' application. 
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LINKING TELECOMMUNICATIONS AND PROBLEM SOLVING 



Dr. Barbara Brehm, Eastern Montana College 

Students In seven middle school classrooms participated in one of three treatment groups to study the 
connection between telecommunications, problem solving training, and the use of problem solving 
strategies in the classroom and over a telecommunications network. Data was collected and analyzed 
using a pre- and posttest and a classroom observation form developed by the researcher. 

Students, participated in the Most Liveable Places Telecommunications Project over the FrEdMail 
Network, by identifying factors which made their city a good place to live, rating their city, and by 
using "receiver site transfer" to analyze information from other cities . Network and classroom 
activities Incorporated levels of problem solving corresponding to Bloom's Taxonomy. 

Findings from the study include significant increases for the CTP fTelecornmunications and problem 
solving) group on posttest scores and on the level of problem solving strategies over the CT 
(Telecommunications) group and the C (Comparison) group. Both the CTP and the CT group also 
generated more problem solving strategies than the C group. 

Recommendations for future research include involving students more directly in sending and 
receiving messages, providing more training for teachers, stressing the problem identification stage, 
and focusing on the "receiver site transfer" stage of the activity. 



ELECTRONICALLY ACCESSING MULTIPLE RESOURCES: NETWORKING 
ISSUES IN MACINTOSH AND MS-DOS ENVIRONMENTS 



Dr. Mary Ann Htndes. The University of Georgia 

When considering the implementation of an information retrieval network, the first decision must be 
to select a hardware environment. This presentation will address the benefits and pitfalls of 
operating in an MS-DOS or Macintosh network environment. Points to be considered will be student 
access and training, the user/computer interface, and cost. This presentation will highlight two 
schools which have implemented information retrieval networks including online catalogs and CD- 
ROM databases. 
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COMPUTER-BASED SPACE PROGRAMMING TECHNIQUES 
TOOLS FOR THE EDUCATOR AND THE PRACTITIONER 



Thomas L. Houser, The University of Tennessee, KnoxviUe 

Interior space programming comprises the collecting and analyzing of data from myriad sources, 
synthesizing information, and preparing reports. Although proven to be successful, conventional 
graphic programming procedures have drawbacks: repetitive, time-consuming techniques; the one- 
time use of visuals: and the redesign of graphics for program reports. The challenge, therefore, lies in 
developing computer-based techniques that encourage thorough data collection, compress the 
timeframe for programming, and reduce redundancy. Procedures must address client-designer 
interaction dynamics, as well. 

The computer-based alternatives presented have been tested In professional practice and the 
classroom. This has been accomplished on diverse projects, Including an industrial research 
laboratory, a day care center, and an educational facility's masterplan. Potential project types run 
the gamut of architectural and interior design endeavors. Field settings range from private meetings 
to sophisticated teleconferences. 

The procedures employ basic drawing, spreadsheet, database, desktop publishing and desk accessory 
programs. The conceptual nature of the drawing programs and the use of macros in the others allow 
quick mastery of the techniques. 

Computer-based programming offers effective tools for producing graphics, analyses and reports. 
They have proven effective In professional and academic settings. With additional application of the 
procedures, designers will acquire expanded repertoires of graphics, thus expediting future projects. 
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GETTING TEACHERS INVOLVED WITH TECHNOLOGY 



Annette Lamb, University of Toledo 

My chalkboard works Just fine. Why do I need a computer? ...I'm six years from retirement Why 
learn something new? ..Sue seen most of the educational software. It's Junk. Computers can't teach. 

These are familiar comments from educators. Regardless of whether they are professors of cherUstry 
or teachers of first graders, many educators are reluctant to embrace computer technology as a tool for 
Instruction. How can we get educators more Involved using emerging technologies (e.g., computers) as 
instructional tools? This session addresses this growing concern. The early Innovators are already on 
the "technology" band wagon. However, what about all the educators In the shadows that aj/en't 
making use of what technology has to offer? Hypermedia applications, desktop publishing, desktop 
presentations and other "high tech" areas are a great way to get educators involved with technology. 
With the use of easy-to-learn authoring and production tools, educators can develop professional* 
quality computer-generated materials. 

This presentation provides sneaky and not-so-sneaky ideas for getting teachers Involved with 
technology. Successes and challenges to achieving these goals are also discussed. Hie session explores 
ways to encourage reluctant educators to use tools of technology for personal, professional, and 
instructional applications. 

It is our job as technology users to encourage the Integration of technology Into the curriculum. We 
often become so entrenched in our daily routines that we forget about the need to reach out to others. 
The session highlights strategies for conducting effective teacher inservlces and workshops Including 
the identification topics for lnservice and workshops, potential problems, as well as, selection of 
formats and activities are discussed. For example, the "one shot" lnservice, courses and workshop 
series, "make-it and take-its," and demonstrations are highlighted. Finally, tips for successful 
workshops are outlined. 



MAKING GEOGRAPHY FUN WITH HYPERCARD 



Annette Lamb, University of Toledo 

This presentation will describe and highlight the development and use of two HyperCard-based 
geography activities. First, Banking Bucks In Brazil is a simulation that incorporates technology 
tools to support student participation in collaborative decision making, information exploration, 
and practice of higher level thinking skills Middle school students participate in a collaborate 
approach to Information exploration through accessing, manipulating, and applying regional and 
Investment information about Brazil. Students have access to regional and investment information, 
compare and contrast regions of Brazil, identify geographical Information, evaluate possible 
Investments, and select a successful investment. This project goes far beyond a simple simulation by 
providing access to a wealth of important information about the country, its regions, and its people 
through a hypermedia format. 

Second the terminology related to geography. Students must create a resort using a random set of 
geographic components. Information about the components is provided to assist users in identifying 
and incorporating these elements into their resort. This game stresses the geographic themes of place 
and location. 
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TECHNOLOGY IN THE CURRICULUM: 
ISSUES AND QUESTIONS FOR THE 90'S 



Dr. Deborah Little, University of Illinois 

This presentation is an interactive exchange of ideas of K-12 classroom teachers, school 
administrators and others interested in technology integration in K-12 settings. Participants will 
share experiences, insights and ideas as the presenter leads them in proposing some possible answers 
to difficult questions. 

The following questions will be the focus of the presentation: 

1. What are some barriers that have made it difficult to integrate technology in the curriculum? 

2. Of the new technologies, which are affordable? Which are not? 

3. How can the unaffordable technology be made more affordable for K-12 schools? 

4. Should teachers be taught to create their own curricular materials using hypertext or other 

authoring tools? 

5. Who should be teaching the teachers to integrate technology in their various curricula? 

6. How do we teach responsible uses of technology? e.g.. Issues of privacy, copyright laws, equity 

7. How or where can teachers and administrators new to technology begin? 

8. What are some successful methods of technology integration in your school or schools you work 
with? 

If time allows, participants will be encouraged to suggest and discuss other issues and questions in new 
technologies for educational settings. 



AN INSTRUCTIONAL MERGER: HYPERCARD AND 
THE INTERGRATIVE TEACHING MODEL 



Carolyn M. Masste, University of Virginia 
Larry G. Volk, University of Virginia 

This paper will explain how one information processing model of teaching can merge with computer 
technology to create a more effective data retrieval process. The hypermedia program on the 
Macintosh computer, HyperCard.will be used to help focus students' attention on the material being 
presented in the lesson. In addition to the hypermedia program being able to Isolate specific data 
display information, using animation and speech can also Increase students' on-task behavior. 

The Integrative Teaching Model combines content information with inductive and deductive 
reasoning to help develop students' thinking skills by having students progress through five 
instructional phases. This paper will summarize the theory behind the model, give reasons for using 
an information processing model, and will point out the advantages of using HyperCard for the data 
display and retrieval. The Integrative Model's five phases are described within a sample lesson plan. 
References will be included which would enable an educator to design hypermedia stacks on 
HyperCard. 
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COMPUTER-ENHANCED LEARNING EDVTRONMENTS 
AND THE REINVENTION OP SCHOOL 



Dr. Kyle L. Peck, The Pennsylvania State University 

To drastically improve today s schools in terms of conventional measures and the development of 
sk SJ s (1I a 4 nd attributes currently overlooked, small-step changes will not do. And. as George Leonard 
said. It may be easier to change all of it than to change any part of it." 

A paradigm shift is needed in education -- a breakthrough on the scale of the discovery of 
microorganisms in medicine, leaving the ground in transportation, or the microprocessor in 
computing. Instructional technology has made almost no difference in the lives of students because 
our approach has been to squeeze technologies in without changing much else. Only when we redesign 
the entire process, with no "sacred cows." will we realize the potential of instructional technologies. 

"Project Rethink! I." set out to "reinvent" education, applying instructional systems design "at the 
macro level. Following a needs assessment, a philosophy of education was developed, and a minimal 
but realtetic set of constraints was identified. Alternatives were synthesized into a powerful student- 
centered learning environment in which active students work in cooperative groups on 
multidlscipllnary. topic -oriented investigations, producing multi-media outputs. An Integrated 
learning system develops basic skills in math, reading, and language arts, and exercises in creativity 
and problem-solving become integral parts of the students' day. 



COMPUTERS AND TEACHER EDUCATION: 
A GRADUATE CURRICULUM DESIGN 



James P. Randall, Ed D„ Widener University 

A unique masters level program in Computer Science Education provides a curriculum design for 
D»tf«Ifi i tea £ her edu r ca »on with emerging technologies at Widener University. Chester 
Pennsylvania. The goal of this program is to infuse classroSm teachers, computer cwrdmators and 
w^h^ 7 w [ th n l ces f com P etenci es. knowledge and value structures to lead the 
integration of computer-based technologies into curriculum and classroom instruction contexts. 

USaSSS^ h 2 Ur ff d l l . ate ? T0 ^ am Provides students with a comprehensive knowledge of computer 
ISSSS? 008 "tJllzation in elementary, secondary and post- secondary education settings In 

fa a coS^nf %L^ 8tUdCnt8 i fr0m a varlety of other « raduatc P r °« rams and thereby allow 
SJLSZ ? ? *. dla l° g betwee . n classroom teachers, computer coordinators and school 
administrators relative to issues of Instructional technology integration. 

ISri^mSI 1 iS . dellVCred m both on - c ampus and off-campus settings. Instructional resources include 
pnv^™„ m h com P" ter 1 Presentation laboratory classrooms which are configured in a LAN 
environment and one Apple development laboratory classroom. Each of the IBM laboratory 

S^JL^TEtSSr WAN envlronment whteh an< ^ 

SSm! ?£& «hX a,y ,n nature address,ng emerglng technolog,es thelr 
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INFORMATICA - A COURSE IN ALGORITHMIC THINKING 



Peter Rizza & Volkert Balk, Princeton Center for Education Services, Inc., 
Alexet Semenov, Institute Jor New Technology 

In 1989 ADCIS sponsored a conference on Computers, Education and Children In Moscow, USSR. At 
that time a small group of participants agreed to create a Joint project between the USA/USSR to 
design and develop courseware based on the Soviet approach to algorithmic thinking and problem 
solving. 

The William C. Norris Institute, MN agreed to fund the Princeton Center for Education Services to 
work Jointly with the Institute for New Technologies in Moscow to produce a new course called 
INFORMATICA, based on this Soviet approach to learning. 

The session will present the instructional designs used in the development of the INFORMATICA 
lessons and will discuss the Soviet approach to the design and development of MicroWorlds. 

The session will present the major advantage of the INFORMATICA course and demonstrate the 
highly interactive use of CBE lessons to develop concepts critical to problem solving. The course also 
provides the students with a fully functional MlcroWorld. A demonstration will show how students 
interact with this open-ended software environment to create algorithms to solve new problems. 

The INFORMATICA course is available to schools and can be obtained from Norris Education 
Innovations, Inc. (NEII) upon request. The INFORMATICA course is offered in a floppy disk as well as 
a networked version. It runs on Macintosh and MS-DOS systems (IBM PS2 & PC-compatibles), and 
will soon be available for MS-Windows platforms. 
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PERSONALIZED EDUCATION MANAGEMENT SYSTEM - PEMS 



Peter J. Rtzza Jr., Princeton Center for Education Services, Inc.; 
Karl G. Kelly, Princeton Center Jor Education Services, Inc.; 
Marilyn Martin, Norris Education Innovations, Inc. 

The Personalized Education Management System™ (PEMS) is a comprehensive multi-user 
application designed to help educators organize and track student progress through a predefined 
curriculum. PEMS™ contains all of the entities, attributes, relationships, and keys in a well deiined 
database structure with a friendly Macintosh interface. PEMS™ specifically supports: 

• Personalized Plan for Development (PPD) a^d Learning (PLP), 

• Curriculum Goals, Objectives, Learning Resources and Tests, 

• Management Plans for Teachers (TMP) and Students (SMP), 

• Planning and Scheduling of instructional Activities. 

PEMS™ allows teachers to prepare student's goals and construct lists of student objectives. PEMS™ 
provides a range of potential goals and objectives and allows the teacher, student, and parent to meet 
to determine the best personalized education plan for the student. 

Teachers can manage the student assignments and scheduling, as well as record student progress by 
clicking on icons in the Teacher's Ofllce. The underlying relational data base allows them to query 
PEMS on-line, with security supported via password sign-ons and group codes. 

PEMS™ calculates and tracks student progress, including test and learning resource scores. PEMS™ 
also allows for manual entry and manipulation of the data by the teacher. 

PEMS™ supports the complete education management cycle, helping professional educators, 
administrators, teachers, parents, and students to move through all the phases in the cycle: 

• Definition Phase - In this phase the curriculum goals, objectives, learning resources, and tests are 
defined, 

• Assignment Phase - Here the appropriate goals, objectives, learning paths, learning resources and 
teste are assigned, 

• Performance Phase - In this phase the students interact with their assigned instructional activities, 

• Progress Tracking Phase - The students' progress is tracked, updated, posted and reported during this 
phase, 

• Evaluation Phase - During this phase the students and teachers can evaluate the system's 
performance and the effectiveness of activities and programs, and 

• Reassignment Phase- Periodically the students are assigned new goals, objectives and learning 
assignments. 

In the presentation we will discuss the design of PEMS 1 ** and the results generated by some of the pilot 
schools that have Implemented PEMS™ within their districts. 



9 

ERIC 



23 

3U 



TRAINING TRACKER 



Peter J. Rizza Jr., Princeton Center for Education Services, Inc.; 
Karl G. Kelly, Princeton Center for Education Services, Inc.; 

The Training Tracker™ is a comprehensive multi-user application designed to help the training 
coordinator organize and track participants progress through a specified training program. The 
application contains all of the entities, attributes, relationships, and keys in a well defined database 
structure with a friendly Macintosh interface, and specifically supports: 

• Planning - professional development assignments for participants, 

• Managing - the participants path through professional development activities, and 

• Defining - the structure and professional requirements within the corporate training environment. 

Training Tracker is a complete data base application that helps training managers document a 
participant's training plan, schedule required correspondence among indiuduals involved in the 
training program, and maintain a complete history of a participant's assignments to the training 
program. Hie application also provides for the generation of overdue correspondence notices. 

Training Tracker provides a list of skills, knowledge, and competencies associated with training 
assignments, indicates which are required of a given Job function and which are appropriate 
development assignments for the participant's job classification. 

Training Tracker provides a means of tracking a participant's progress against the skills, knowledge, 
and competencies for an assignment, and maintains a complete history of the participant's progress 
through the training program. 

Training Tracker provides multi-user access to the application and secure access, by individual and 
group, to features and information. 

The session will present the overall structure of the application, give examples of how it can be 
implemented and demonstrate the functions and features. 
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A CONTROLLED STUDY ON DEVELOPMENTAL 
WRITERS AT THE COMPUTER 



Gary Vaughn, University of Cincinnati 
Rex Easley, University of Cincinnati 

Ihe 1990-91 academic year marked the fifth and final year of a study at the University College of the 
University of Cincinnati to measure the effects of classroom computer use on the composing processes 
of developmental writing students. Five English instructors and a large number of sections of a 
developmental writing course (a total of over 1,000 students) were involved in the study. The freshman 
students are placed into this course on the basis of their performance on a holistically-graded essay 
and on a standard reading test, They have the opportunity to retake the essay test at the end of each 
quarter; a passing grade is needed before they may enter into the Freshman English sequence. 
Sections Involved in the study used either the Bank Street Writer II word processing program, or Bank 
Street Writer in combination with William Wresch's Writer's Helper writing-aid program, or no had 
no computer usage at all. All classes in the study used the same syllabus with the same assignments 
and the same point-based grading system. insults form the five-year study were subjected to 
statistical analysis not yet completed at the time of this proposal submission, However, preliminary 
data indicate that none of the three groups performed at a significantly higher level on the essay test 
or averaged a significantly higher quarter grade. On the other hand, when the results are examined 
from a perspective of traditional students versus non-traditional adult learners (about ten years older 
on the average), the adult learners passed the essay exam at a much higher rate if they belonged to 
sections which used Writer's Helper . Furthermore, student evaluations indicated a positive response 
toward using the computers, showing, for example, that students in the computer classes were more 
likely to Judge their writing assignments as "interesting and stimulating." Complete results of the 
study and its ramifications will be discussed. 



SCIENCE INSTRUCTION ON GLOBAL ELECTRONIC NETWORKS 



Michael L. Waugh, University of Illinois at Urbana-Champaign 
James A. Levin. University of Illinois at Urbana-Champaign 

Real world science is conducted in global contexts, with collaboration among colleagues around the 
world a common occurrence. With the spread of microcomputers into science classrooms and the 
availability of electronic networks, it is now possible to conduct science instruction in such global 
contexts as well. This presentation will discuss ongoing research and development efforts concerning 
ways In which electronic networks can be used In science teaching and learning. Electronic networks 
can allow students to participate in distributed science problem solving activities by providing 
"dynamic support" so that students can gather and analyze data and Jointly create project reports with 
students from other parts of the world. Several of these "telescience" activities will be described In 
detail. In one activity, students from across the United States have been engaged in data collection 
and data analysis activities related to estimating the circumference of the earth. In another activity, 
students and teachers participated in the collection and analysis of data concerning the cost, 
nutritional value and appeal of typical school lunches. The session will conclude with a discussion of 
the potential for electronic networks to Improve science teaching and learning. 
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REALIZING THE POTENTIAL OP A LEVEL ONE VIDEODISC-BASED 
SCIENCE PROGRAM POR ELEMENTARY CLASSROOMS: 
A REPORT OP AN EVALUATION STUDY 



Gayle V. Davidson, PhD., The University of Texas at Austin 
James L Barufaldi, PhD., The University of Texas at Austin 
Lowell J. Bethel PhD., The University of Texas at Austin 

The purpose of the evaluation study was to ascertain the effectiveness of the videodisc -based 
elementary science program in selected fourth grade classrooms as determined by the users of the 
program Involved In the pilot testing. This study Is qualitative In nature, supplemented with 
quantitative information. Student Information was collected through the administration of 
interviews, assessment of science- and technology- related attitudes, skill? and content, and 
classroom observations. Information about the pilot teachers was collected through demographic 
surveys, Interviews, classroom observations and administration of assessments of concerns and 
anxiety towards science and technology. The selected schools also represented the socioeconomic and 
ethnic diversity of Texas, the study was conducted to : (1) describe classroom behaviors and 
perceptions of students and teachers and the ecology of the learning environment, (2) determine 
teachers' attitudes toward technology and state anxiety toward the teaching of science, (3) identify 
students' attitudes toward science/technology and Interest in science outside the classroom, and (4) to 
identify students' science content knowledge and skill development based before and after the 
implementation of the videodisc-based science program. The outcomes of the pilot test show promise 
that the videodisc -based program is effective in initiating positive changes among teachers and 
students towards science and technology. Data supported the notion that the program is effective in 
the fourth grade pilot classrooms. The presentation will report and discuss the findings of this 
evaluation study. 



OBJECT ORIENTATION AND THE DEVELOPMENT OP 
COMPUTER-BASED INSTRUCTIONAL SYSTEMS 



Eric Hansen, Educational Testing Service 

Object orientation is a software engineering methodology that underglrds many of the recent advances 
in authoring productivity tools, graphical user interfaces database technology, distributed processing 
multimedia, and lnter-application communications. The key concepts of object orientation - objects 
classes, and messages - will be introduced. The considerable benefits of object orientation for 
developing computer-based instructional systems will be discussed, particularly for multimedia 
applications. 
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THE APPLICATION OP HUMAN FACTORS TO THE ID 
PROCESSES FOR COMPUTER-BASED PROGRAMS 



Hueyching Janice Jlh and Michael A. Orey, 
The University of Georgia 

The field of Instructional design (ID) integrates knowledge, ideas, techniques, and technology from a 
number of areas, for instance cognitive science, learning theory, media technology, and computer 
science The advent of computer-based instructional materials as a new kind of instructional vehicle 
brings a considerable challenge to the education/training field. The appeal of computer-based 
instructional (CBI) materials, with its myriad opportunities for instructional innovation and 
creativity, has encouraged iD professionals to push ahead. We suggest here that advances in human 
factors will provide fruitful ideas for developers of CBI. 

For the past two decades, the field of human factors has been contributing greatly to the design and 
development of computer-based technology. Specific questions that will be addressed in the 
presentation are: 1) What new directions do we need to pursue to improve current mindsets to 
instructional design for computer-based materials?; and. 2) What efforts to human factors field might 
contribute to the modification of a five phase (analysis, design, development, implementation, and 
evaluation) phase model of ID for computer assisted instructional systems? The purpose of this 
presentation is to indicate some salient aspects of human factors research which can shed light on 
instructional design of CBI. The focus of this discussion will be towards designers and developers of 
CBI. 



HYPERMEDIA IN THE INFORMATION DISSEMINATION PROCESS 



EUen N. Piety, U. S. Army Construction Engineering Research Laboratory 

How can an Army Research Laboratory disseminate information on all their ongoing research 
projects, either fielded or still under development, in a convenient and timely manner? This is a 
period of minimal time for fact absorption, complex innovative technological advances, varying 
levels of technical expertise, multi sensorial output devices and varying individual human 
nformatlon processing skills. Which are the best technologies to transfer critical knowledge 
optimally, efficiently and yet to as short a time as possible. 

Hypermedia seems to offer the optimal solution to this variety of challenges with lis unique and 
enriching experiences, any person can have a significant interaction with a hypermedia information 
dissemination system. The design, implementation and validity of nformation presented needs to be 
carefully constructed and flexibly delivered to meet the potential users' needs. 

Current research underway at USACERL is investigating the applicability of this emerging technology 
to the area of nformation dissemination. Major objectives include analysis of the best techniques for 
different types of presentation, optimal branching capabilities, ease of knowledge acquisition, and 
permitting the user to easily find his way through the levels of nformation he is interested to, without 
getting lost in "hyperspace". 
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COGNITIVE OUTCOMES OF BUILDING EXPERT SYSTEMS 



David H. Jonassen, University of Colorado 

Researchers and businesses i>re Investigating the effects of having students and employees construct 
expert system rule bases. What Is Important are the cognitive effects of the construction process more 
than the resulting advisors. E rpert systems can be thought of as cognitive tools engaging learners in 

2S^^ n ? ta £8? ltttve rt Sr a V outcomes - 11118 Project proposes to investigate the changes in 
critical thinking skills, cognitive learning strategies, and metacognitive skills that result from 
employees and students constructing rule bases. Changes In individuals' knowledge structures will 
also be investigated. 



USER'S PERFORMANCE AND PREFERENCE ON DIFFERENT 
INPUT DEVICES: TOUCHSCREEN, MOUSE AND KEYBOARD 



Chi-Hui Lin & Gayle V. Davidson, PtiD. 
The University of Texas at Austin 



This presentation begins with a discussion for the consideration of human factors on designing 
computer-assisted instruction with respect to Input devices. The three most frequently used input 
devices (touchscreen, mouse, and keyboard) in educational settings will be discussed. The discussion 
will Include the advantages and disadvantages of using each of theses three devices based on both the 
users and the developer's points of view. The presentation will culminate with a reporting of 
SSS 10 ? r< ? earch on the , e f «*? of the use of these input devices on task perform ance and attitudes. 
Handouts of summary points of the presentation along with references will be provided 



INTELLIGENT TUTORING WITH HYPERTALK: WHO NEEDS LISP OR PROLOG 



Wayne A. Nelson, Southern Illinois University at Edwardsville 
Safiah MdYusof, Southern Illinois Untverstty at Edwardsville 

LIS 11 ? 6 ? tu f torlnfl systems (ITS) represent a new approach to instructional software design that takes 
? *f C u r f Cent advan ? es 5 cognitive science research. Rather than making aU decisions about 
instruction before presenting the software to the learner, ITS utilize representations of leura and 
expert knowledge to diagnose learner errors and select instructional tactics and curriculum sequence 
in a dynamic process which unfolds as the learner Interacts with the system Such an apSmS 

'ifftS ^n«£ Pt , ° W"* 1 P erfoma " ce ln ways not possible for software deveSpeS 
using the traditional Instructional systems approach to design. 

Object-oriented systems such as HyperCard provide a viable alternative for imnlemenHn* 
histructional software which utilizes some ITS techniques. The pr^^l^ J^ 1 ^^ 
SEES te ? h ?** ues for tutoring in HyperCard related to interfacT design J^ate 

structures for knowledge representation, diagnosis of errors, instructional transactions and 

dSSStot ^ 3 !qUeI,Ctag • A SP 6X31111)16 ° f 3 SyStCm P rofirammed entire* in hSSSS fwilf be 
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WHY AUTHORING TOOLS SHOULD BE OBJECT-ORIENTED 



Joseph Seeley, University of Illinois 

Object-oriented programming (OOP) can increase programmer productivity in several ways. It 
supports the rapid creation of new software components out of old ones and the relatively easy 
integration of new and old. It makes possible reliable, reusable components with which the designer 
can compose new applications. It factors code in ways that can lessen both the impact of changing 
requirements and the extent of maintenance needed. 

Three aspects of OOP lead to the above benefits: data abstraction, Inheritance, and polymorphism. 
Understanding these concepts allows courseware developers to understand how OOP can help them. It 
also helps them separate object-oriented hype from substance. 

Courseware authoring differs from other software engineering tasks in several important ways. 
Among these are the vital importance of the interface and the Inherent difficulty of manipulating 
mental processes instead of data. These differences suggest that OOP may be an especially appropriate 
tool for authors. 

Examples taken from an object-oriented framework for drill construction illustrate the presentation 
throughout. 



THE COLORADO LITERACY NETWORK: HOW AN 
ELECTRONIC COMMUNICATION NETWORK ENHANCES 
THE ADULT LITERACY PROGRAMS 



Eric E. Smith, University of Northern Colorado 
Jennifer G. Rudd, EG&G, Rocky Flats 

This paper addresses the communications problems of adult literacy providers dispersed over a large 
geographical area. The need for increased communication to provide better services and the 
implementation of an electronic network to facilitate communication are discussed. 



OBJECT-ORIENTED PROGRAMMING: A PASSING 
PAD OR THE COMMING OP A NEW GENERATION 



Jennifer A. Chen-Troester, Utah State University 
Zhongmin LU University of Utah 

Object-oriented programming (OOP) is becoming the programming methodology of the 1990's. 
Through the use of an object model, encapsulation, abstraction and inheritance, object-oriented 
programming enables programmers to easily reuse, extend and modify code. We will discuss the basic 
elements of OOP, how object-oriented design is different from traditional procedural software design 
and also look at designing a project using two object-oriented tools, HyperCard and ToolBook. 
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A CASE STUDY IN DISTANCE DELIVERY 



Michael Yacct, Rochester Institute of Technology 

An introductory graduate college course in Learning Theory was distance delivered to a Fortune 500 
company In upstate New York. Prior to distance delivery, the course had contained elements of 
Personalized System of Instruction (PSI), along with elements of a "traditional" college class, such as 
student discussion and activities. Although PSI has components that are easily adapted to distance 
delivery, the adult, professional audience still expected learner-Instructor interaction and discussion. 
Therefore, a hybrid mediated course was designed that could meet the requirements for both student 
autonomy and interaction among learners. 

The course objectives specify that the students will learn a large amount of declarative knowledge 
coupled with some specific procedural skills. This means that the course primarily teaches verbal 
lnfonnation intermixed with some specific concepts and principles. Its Intent Is to supply the learner 
with the background knowledge for other courses in the curriculum by creating an interrelated 
cognitive structure of various learning theories. By the end of the course, students are expected to see 
similarities and differences between competing theories as well as use and apply these theories to a 
limited degree. 

The instructional events of presentation, elaboration, practice with feedback, and testing were 
approached by analyzing the cognitive activities necessary in the learners and then by carefully 
selecting a strategy and a medium that could encourage and support these activities. This translated 
Into each unit of the course being comprised of required reading, audio and video tapes, a work book, 
handouts, worksheets, experiential activities, and electronic mall. 
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ADAPTING HYPERCARD STACKS TO PROVIDE SPEECH 
OUTPUT FOR READERS WITH DISABILITIES 



Wren M. Bump, Texas A&M University 

HyperCard stacks can be enhanced with sound and speech capabilities. Do you want to use "real" 
speech or "machine" speech? What Is your purpose for the sound and how much memory do you have 
available? How are both types of sound installed into the stack? What does the student with special 
needs require in order to be successful in the use of the HyperCard stack? These questions will be 
answered In this presentation. 



ALLOWING STUDENTS WITH PHYSICAL DISABILITIES ACCESS TO 
COMPUTERS THROUGH ADAPTIVE TECHNOLOGY 



Wren M. Bump, Texas A&M University 

Students who do not have the physical capability or control needed to use computers In the traditional 
way are many times not given the opportunity to use computers at all. These are the very students who 
could benefit the most from computer use. Adapted keyboards and switches will be shown that help a 
range of students. 
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THE IMPACT OP COMPUTER-BASED LEARNING ON EXAMINATION 
PERFORMANCE IN MEDICAL BIOCHEMISTRY 



James Baggott, PhD and Sharon E. Dennis, M.S., 
Hahnemann University 

An optional student-controlled computer-based learning (CBL) environment (lecture, review, self- 
testing and clinical problem modules) covering over 50% of a Medical Biochemistry course was made 
available to first year students. Users experienced Improved performance on multiple choice 
examination questions. The percentage improvement on questions for which CBL was used is 
predicted by the intensity of CBL use. 



THE DEVELOPMENT OF A REPURPOSING VIDEODISC WORKSHOP 
FOR HEALTH PROFESSIONAL FACULTY 



Michael Weisberg, Ed.D., National Library of Medicine 
Victor Carr, Ed.D, National Library of Medicine 

This presentation describes the concepts and developmental process involved in the creation of a two 
and one-half day "hands on" workshop that enables health professionals to repurpose existing 
videodiscs. The ideas for the workshop evolved from staff interaction with health professionals at 
The Learning Center for Interactive Technology and were first tried out in the format of brief ( two and 
one-half to three hour ) seminars at national health professional meetings held throughout the 
country. These seminars provided an introduction and overview of the concept "repurposing", and 
were elucidated with several concrete examples. As staff gained knowledge and experience working 
with videodiscs, the seminars evolved into a plan for developing a "hands on" workshop where faculty 
would design and create repurposed videodiscs. This plan involved the systematic development of a 
full-scale workshop. Forty- eight health professionals participated in four developmental workshops 
conducted by the staff of The Learning Center for Interactive Technology. ( In addition to the 
presenters of this session, major contributions to the workshop were made by Richard Banvard, M.A , 
Craig Locatis, Ph.D , and Mr. Quang Le). 



9 

ERIC 



35 51 



HEALTHY PEOPLE 2000: OPPORTUNITIES POR COMMUNICATION 
TECHNOLOGY IN DISEASE PREVENTION AND HEALTH PROMOTION 



Maiy Jo Deertag, Ph.D., Office of Disease Prevention and Health Promotion 

The biomedical science base supports health promotion and disease prevention efforts to reduce 
disease, disability, and premature death. Escalating medical costs encourage informed decisions 
about the appropriate use of preventive and other health care services. Healthy People 2000: National 
Health Promotion and Disease. Prevention Objectives was developed as a national consensus 
document under the leadership of the U.S. Public Health Service. The 300 objectives target 
improvements in health status, reductions in risk, and expansion of services. They are grouped In 22 
priority an as ranging from specific chrome diseases such as cancer to nutrition and physical 
activity. 

Information and education activities developed by or for health care professionals, health officials, 
schools, worksites, national and community based organizations, and the individual consumer will 
be important tools for achieving these objectives. Technological applications about preventable 
health problems can facilitate long-term behavior change through participatory learning. 
Interactive video, utilizing scenes from real or staged patient counseling sessions, can help teach 
medical students-or doctors-how to counsel their patients about behavioral risks such as smoking, 
poor diet, or exercise. Decision-support mechanisms can help professionals, patients, and consumers 
sift through an ever-increasing prevention knowledge base. 

This presentation, with a video and slides, will outline the opportunities for computer-based 
educational products and services within the framework of Healthy People 2000. 



DEVLOPMENT OP COMPUTER EDUCATION IN HEALTH SCIENCE 
(NURSING): SOME ALSTRALIAN EXPERIENCES 



Dr. Nttya Karmakar, University of Western Sydney (Hawkesbury) 

University based nursing education is new in Australia. In New South Wales, diplomas in nursing 
started in 1985 and universities are now offering bachelor degrees. Computing has not been 
significant in these rapidly developed curricula. This paper looks at Important Issues in Introducing 
computers in nursing education, because the nursing profession in Australia Is challenged by changes 
in information technology. 
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THE NEGLECTED ROLE OP EVALUATION: 
INSTRUCTIONAL PRODUCT SELECTION 



Marjorte A. Kuem, PtiD., National Library of Medicine 

Evaluation-its definition, approaches, purposes-has received a lot of consideration in professional 
literature in the last 2 decades (See, e.g., Scrtven (1967, 1972, 19741, Stufflebeam 11971, 1974], Airasian 
119741). Two purposes of evaluation that appear most frequently in its definitions are (1) to render 
Judgment on the worth of a program; and (2) to assist decision makers. 

Assessing the worth of a program is a concept basic to evaluation. However, early in the history of 
modern evaluation theory, evaluation was distinguished from actually rendering judgments (See 
Alkin (1969), Patton (19781, Stufflebeam and Webster (19801, and Weiss (19721); that was 
the role of the decision maker. In this age when instructional materials are developed by commercial 
firms, other educational Institutions, or enterprising individuals, often the roles of evaluator and 
decision maker belong to the same person, whose responsibility it is to determine the value of 
instructional products and to decide whether to Incorporate them into curricula. 

Frequently, evaluative information is found in descriptions of educational products provided by 
developers or disseminators; and evaluation is included as a sub-topic in presentations regarding such 
programs and products, specifically to sway decision makeis who consider adopting the products. 
Curriculum decision makers can approach their product selection tasks more confidently if they have 
compiled basic data about the course or program for which they are seeking a computer or other 
technology-based product. These data serve the critical role of guiding their product review and 
selection activities. 

The presenter will provide guidance for what course/program and technology data are most relevant 
and how to navigate through and evaluate the information provided about a product. 



INTERACTIVE VIDEODISC FOR NURSING EDUCATION: VITAL SIGNS 



Donna Larson, Grand Valley State University 
Barbara Breedon, Fuld Institute for Technology in Nursing Education 

Design, content, and production issues will be discussed during the demonstration of this interactive 
videodisc series currently under development at the Fuld Institute for Technology in Nursing 
Education. The content (the measurements and interpretation of temperature, pulse, respiration, and 
blood pressure) reflects development of both psychomotor and cognitive skills in beginning level 
nursing students. 
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A DATABASE AND STUDY OP AUTHORING SYSTEM TECHNOLOGY 



Craig Locatis, National Library of Medicine 

A review of authoring systems was undertaken to update an earlier study of the software conducted by 
the National Library of Medicine in 1985. A literature review was made and an entirely new review of 
available products was conducted because the technology and marketplace had changed significantly. 
Over 170 potential products were identified, but only 82 of these were actually reviewed because many 
are 1) not being marketed. 2) not truly creating computer-based instruction, or 3) running on 
mainframe or older microcomputers (e.g.. Apple H's) not common in medical schools. A number of 
trends, like more graphical interfaces and multimedia capability, were identified. Reviewers were also 
able to detect varied vendor approaches to the market. Of particular interest are new "intelligent" 
authoring tools and more objective selection methods. The study explores potential problems in using 
the new tools and evaluation techniques. AuthorBase was developed in support of the study in 
anticipation that future reviews will be likely. Since the database contains source, hardware, cost, and 
other information about specific products, it may be useful to others. 



AN INTERACTIVE TECHNOLOGY SAMPLER 
VIDEODISC IN THE HEALTH SCIENCES 



Craig Locatis, National Library of Medicine 

The Interactive Technology Sampler portrays twenty interactive technology applications in the 
health sciences that can be viewed at the National Libr of Medicine's Learning Center for 
Interactive Technology. The disc is intended to make infounatlon about the technology available 
outside the center and Is designed for Individual use in kiosks and exhibitions and in formal 
presentations conducted by center staff. In addition to the portrayals, there are operating 
instructions, an attract loop, and "tours" providing supplemental information about some programs 
on the second soundtrack. An Index allows accessing programs by subject, title, and application. 

The disc Is designed for multiple levels of use: level 1 (player keypad), level 2 (on board program), and 
level 3 (computer control). The current level three programs allow interaction with a single 
touchscreen on IBM InfoWindow or mouse using M-Motion or dual screen display with IBM PCs or 
Apple Macintosh computers. Barcodes also can be used for access. The design maximizes the disc's 
functionality, both with or without external computer control, and uses video production techniques 
that shorten presentation length and Increase presentation rate. 
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A COMPUTER-BASED APPROACH TO REINFORCING 
COMMON PRINCIPLES IN BIOLOGY 



Harold Modell National Resource for Computers in Life Science Education 

Students cften approach each course in a curriculum as a body of completely new information that 
has little in common with previous or future courses. In doing so, they fail to realize that our 
understanding of biological phenomena Is based on a set of basic principles that are applied in a 
variety of ways. To help introduce students to underlying principles and make them aware that these 
principles can be applied in a variety of ways, we have begun to develop a series of computer exercises 
focusing on common themes in mammalian physiology and designed for use in an Independent study 
setting. The first program in the series deals with the principle of conservation of mass applied to 
Indicator dilution determinations and mass balance analyses in cardiovascular, respiratory, and 
renal physiology. The program contains aspects of the three approaches to computer-based education; 
tutorial, drill and practice, and simulation. The overall context of the exercises is that of a series of 
simulated experiments, all requiring some aspect of a conservation of mass analysis for determining 
the value of a specific parameter. In defining the specific experiment, the student provides descriptive 
data about the subject or conditions of the experiment. From this description, the program calculates 
values for the physiological parameters ("unknown") within the system being tested. Thus, the 
simulated experiment leads to a drill and practice problem in which the student must calculate the 
value of the "unknown". Each drill and practice problem contains appropriate tutorial feedback for 
reinforcing how conservation of mass is applied to the problem, should that be necessary. 



AN OSMOTIC PRESSURE TUTORIAL THAT 
REINFORCES REASONING SKILLS 



Harold Modell National Resource for Computers In Life Science Education 

We have developed a computer based tutorial to help students review the basic principles underlying 
osmotic pressure ~nd osmotic relationships within physiological systems. However, unlike the 
tradional question/answer tutorial format, the design of this program is intended to reinforce 
scientific reasoning skills by having the student pose and test hypotheses. The program begins with a 
demonstration of water movement in an osmometer after a 1% sucrose solution is added to one side of 
the semipermeable membrane. The next several screens ask the student what took place, explain the 
phenomenon, and define osmotic pressure. The student is then asked whether particle size, particle 
number, or both determine the osmotic pressure. No information regarding the answer to this 
question L« presented In previous screens. Thus, the student must form an hypothesis. The program 
then allows her to test this hypothesis using a simulated osmometer with either several choices for the 
test molecules or the concentration of the test solution placed on one side of the osmometer 
membrane. Upon completing the experiment, the student may choose to run another experiment or 
choose to stop gathering data. If at least tw^ experiments have not been run, the program reminds the 
student that not enough data has been gathered to test the hypothesis. When sufficient data have been 
obtained, the program asks if the data support the proposed hypothesi ■ The program response 
provided following the correct choice reinforces the observed results rather than merely restating the 
correct answer. The tutorial covers basic principles of osmosis, tonicity, and the interaction of 
hydrostatic and oncotic pressures determining fluid exchange in the capillary. 
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CAI: A LEARNER VIEWPOINT 



U Q. The.de, Kent State University 

In many settings, getting college nursing students to use CAI is a challenge. This despite studies in all 
fields which indicate that students learn as much with computer instruction as with so called 
"traditional methods," and often in less time. Given these facts and today's time conscious student, it 
would seem logical for students to be calling for more CAI, not resisting its use. 

Insight into this situation was gained through voluntary interviews with nursing students who used 
one of four computer programs, tutorials about blood gases or dosage calculation, a simulation, or a 
nutrition analyzer. The results Indicated that interactivity, or the ability to respond to learners, 
which many see as the key to CAI effectiveness, goes beyond the expected software learner 
combination to the learner's feelings about computers and beliefs about learning. 

Three overall categories of Interaction influence were found: 1) factors based on individual beliefs 
about learning, 2) factors derived from learner interactions with the computer and computer 
environment, 3) interactions with the software. This presentation will describe each of these 
categories and illustrate them with quotes from the interviews. Besides pinpointing problem areas, 
much of the data suggests positive actions that can increase CAI use. 



PANTHERS AND LIONS: TRANSFORMING 
COMPETITION TO COLLABORATION 



Karen S. Wedge, University of Pittsburgh 
Dee McGonigle, Pennsyl'xmia State University 

Two faculty researchers at traditional football rival universities present the trials, tribulations, and 
ultimate successes of collaborating on a Joint developmental research project. Collaborative projects 
present challenges In searching for a collaborative partner, melding research protocols, and logistics 
of project development. 

The presentation includes a description of the collaborative project - an Interactive video program 
entitled "Perinatal Data Base and Tutorials: An Elaborated Resources Approach to an Interactive 
Video Hypermedia Application." This project encompasses the three functions of computer based 
learning as suggested by Taylor (1980): tutor, tool and tutee. As a tutor, the multiple mlcrotutorials 
related to perinatal data provide access to the body of knowledge contained In the program. As a tool, 
the project produces an extended visual resource for learners that interconnect in an elaborated 
resources program including access to an on-screen glossary from any point in the program. As a 
tutee, the project provides a lesson shell that enables learners to design units of self study or visual 
presentations from the videodisc and actually extend the program. The project's flexibility supports 
use by undergraduate students initiating inquiry into perinatal Information as well as by graduate 
students synthesizing theoretical concepts or by others needing to review or present this material. 
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BUYING CLOTHING SERVICES, SUPPLIES, NOTIONS, 
EQUIPMENT AND PATTERNS BT MAIL 



Bette Jo Dedic, University of Kentucky 

The MoreMail computer program is designed to be used as a resource for home-based sewing 
entrepreneurs, educators, home sewers and other consumers in locating mail order items needed for 
the design, construction, care, and maintenance of clothing, textiles and related articles. These items 
Include sewing notions, supplies and equipment, patterns, and related services. Over 200 businesses 
are represented in the data base. 



HOME ECONOMICS, SOCIAL STUDIES, AND HEALTH TEACHERS' 
ATTITUDES TOWARD AND USE OP COMPUTERS FOR INSTRUCTION 



Chryssoula Drizou, Iowa State University 
Cheryl Hausqfus, Iowa State University 

The purpose for this study was to investigate the differences between secondary teachers teaching In 
the social sciences on availability, use, and attitudes towards computers. In particular, this study 
focused on three areas of the social sciences where similarities on the Implementation of computers 
were anticipated: home economics, social studies, and health studies. 

A stratified random sampling method was used to identify the Invited sample of 201 teachers with 67 
teachers selected from each group. Data for this study was obtained by a questionnaire mailed during 
winter 1989-1990. The instrument consisted of three parts: demographic information, attitudlnal 
scale, and open-ended questions. The attitudlnal scale was designed to describe three types of 
attitudes: a) anxiety of using computers b) like working with computers, and c) confidence in ability to 
use computers. 

Data analysis showed that the majority of teachers (61%) reported using the computers in their work 
but only 41% of them use computers for instructional purposes. A greater proportion of home 
economics teachers (62%) report using computers with their students than social studies (22%) and 
health teachers (47%). Teachers indicated that they do not feel anxious or fear computers; rather, they 
like using computers but they do not show significant confidence in using computers in their 
classrooms. 
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COMPUTER-BASED SPACE PROGRAMMING TECHNIQUES: 
TOOLS POR THE EDUCATOR AND THE PRACTITIONER 



Thomas L. Houser, The University of Tennessee 

Interior space programming comprises the collecting and analyzing of data from myriad sources, 
synthesizing Information, and preparing reports. Although proven to be successful, conventional 
graphic programming procedures have drawbacks: repetitive, time-consuming techniques; the one- 
time use of visuals; and the redesign of graphics for program reports. The challenge, therefore, lies In 
developing computer-based techniques that encourage thorough data collection, compress the 
timeframe for programming, and reduce redundancy. Procedures must address client-designer 
Interaction dynamics, as well. 

The computer-based alternatives presented have been tested In professional practice and the 
classroom. This has been accomplished on diverse projects, Including an Industrial research 
laboratory, a day care center, and an educational faculty's masterplan. Potential project types run 
the gamut of architectural and Interior design endeavors. Field settings range from private meetings 
to sophisticated teleconferences. 

The procedures employ basic drawing, spreadsheet, database, desktop publishing and desk accessory 
programs. The conceptual nature of the drawing programs and the use of macros in the others allow 
quick mastery of the techniques. 

Computer-based programming offers effective tools for producing graphics, analyses and reports. 
They have proven effective in professional and academic settings. With additional application of the 
procedures, designers will acquire expanded repertoires of graphics, thus expediting future projects. 



ELUCIDATING TIME MANAGEMENT ACCOUNTABILITY 
THROUGH SPREADSHEET ANALYSES 



Thomas L. Houser, The University of Tennessee 

In service professions such as architecture, engineering and Interior design, fees are determined on an 
hourly or project basis. Either base challenges the professional's time management accountability. 
Firms maintain detailed records to vouch for efficiency and effectiveness and to predict requisite staff 
commitments for future projects. Accountability is critical to the individual and the firm's success. 

The challenge for the educator lies in cultivating the student's sensitivity to time management 
principles. To address this challenge, computer-based spreadsheet exercises have been developed to 
record and analyze time usage. These timesheet projects have been used in microcomputer 
applications and business procedures classes. Students delineate coursework and employment 
activities for a minimum of three months, entering this information into spreadsheets beginning 
midway through the study. They then extract and make anah/ses of pertinent data. Students also 
prepare tables and graphs by using the application's chart commands or by running macros developed 
for the project. Representative analyses include comparisons of time spent on academic and non- 
academic endeavors; distribution of efforts among tasks associated with specific courses; patterns of 
effective time management displayed throughout each week and over the term; and percentages of 
effort relegated to individual commitments. 

Through course evaluations, students have indicated these exercises to be beneficial. Findings reflect 
enhanced sensitivities to accountability and improved effectiveness in time management. 
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REPACKAGING: ONE KEY TO COST-EFFECTIVE TRAINMNG 



Elizabeth Miller, Control Data Corporation, 
Cheryl Shankman, Hughes Training Systems 

Frequently companies reject "canned" training materials with the cry, 'That's nice, but that's not what 
we do!" Upon closer inspection many of these programs contain 80 - 90 percent of the necessary 
training, but are missing the 10 - 20 percent that customizes it for a company or department's specific 
needs. Using real-life examples from DoD, industry, and manufacturing, the presenters will discuss 
their experience with "repackaging" existing materials and/or developing new materials to provide 
cost-effective training solutions. 

Repackaging provides companies with a series of advantages, including: 

• the ability to customize and reorganize pre-existing courseware to make a new curriculum to 
meet your organization's specific needs. 

• the ability to lntegtrate non-CYBIS courseware into curricula. 

• the ability to add placement tests, increase question test banks, change testing strategies and 
scoring procedures, add feedback, etc. for existing courseware. 

The presenters will also discuss repackaging techniques utilizing the ISD method to insure 
instructional effectiveness and systematic methodology in the design and development of new 
training materials and the integration of existing materials including print, CBT, audio, video, 
reference, and lab materials into the new curriculum. 



A QUALITATIVE ANALYSIS OF STUDENT TEACHERS 1 
CLASSROOM MANAGEMENT TECHNIQUES 



Cecelia Thompson and Dale E. Thompson 
The University of Arkansas 

Student teachers' in home economics, business, and industrial education kept diarys chronicling 
their positive and negative experiences with classroom management over a twelve week period 
Students were asked to write a weekly entry in response to each of these two open ended statements. 
This week, I handled this discipline problem most successfully by reacting in this way. I handled this 
discipline problem least successfully by reacting in this way. They described both the discipline 
problem and their methods of management. 

HyperQual, Macintosh software designed to facilitate qualitative data analysis, was used to analyze 
diary entries. HyperQual is an application of HyperCard that allows researchers to search text for 
materials relevant to a study. The researcher can code chunks of data and sort them into categories. 

Using HyperQual. meaningful topic categories were isolated to identify similar problems faced by 
student teachers and successful and unsuccessful strategies employed by student teachers. Problem 
solving techniques were identified and classified. 

5 I 19 ?,? Hype*®" 01 Version 3.0 (Computer Program). Chandler, AZ: Author. (Address: 
3327 N. Dakota. Chandler, AZ 85224 U.S A) *«uuicw>. 
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CADD IN THREE DIMENSIONS: EXAMINATION AS DESIGN 
TOOLS FOR INTERIOR DESIGN PROGRAMS 



Susan Winchip, Illinois State University 

Computer-aided design and drafting (CADD) has penetrated all phases of the Interior design industry. 
Now the field has expanded to a score of programs with 3-D graphics software. Some of these 3-D 
graphic programs are aimed at very specific markets, while others have wide applicability. If a 
designer needs to create realistic images in three dimensions, it becomes essential to understand how 
3-D programs work generally, and the strengths and weaknesses of the currently available software. 
The objectives for the presentation and the participants are to 1) Explore the variations of 3-D images, 
including wire frame, hidden line, shaded, ray traces, and antialiasing, 2) Examine the methods for 
ere ting 3-D models, Including coordinate input, 3-D primitives, curved 3-D surfaces, and multiview 
ir t, and 3) examine the series of operations whlcn basic Image-creating methods require including, 
revolving, extruding, surface sweeping, connecting of cross sections, reshaping, rotating, translating, 
and zooming. This presentation focuses on what 3-D in perspective is and how it can create effective 
realistic images for interior designers. In addition, two architectural programs will be compared and 
contrasted. One program is architectural rendering which is a visualization tool that allows a 
designer to create three-dimensional views of projects, but not working drawings. The second program 
is an integrated architectural design system which allows a designer to produce fully-dimensioned 
working drawings, elevations, cross sections and bills of material. The interaction between a three 
dimensional program and its users can be influenced by both the system itself and the participation of 
the users., Thus, in order to Increase the user's satisfaction, performance, effectiveness, and to create 
high-quality drawings it is critical to build a good understanding of the variations in a 3-D images, the 
methods for creating 3-D models, and the basic operating methods. This presentation serves to 
enhance the interaction between three-dimensional programs and the interior design process. 
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TOOLBOOK — SPEAKING FROM EXPERIENCE 



Lorinda Brader, Perm State University 

CBEL— Teaching and Learning Technologies Group of Penn State supports faculty in the use of 
interactive technologies in teaching and learning. While assisting faculty in finding, evaluating, 
using, and developing instructional software, our unit has worked in depth with several programming 
systems including HyperCard and, most recently, ToolBook. This presentation will highlight 
advantages, disadvantages, problems, and solutions that we have encountered in our experiences with 
ToolBook. 

As with any new product, people are interested in learning how ToolBook can be used to solve 
instructional problems. They are interested in knowing how easy ToolBook is to learn and what 
problems they may expect to encounter while developing a product. This presentation will focus on 
the use of ToolBook in an instructional environment. A description of the problems that were 
encountered while learning and developing in ToolBook and how these problems impacted 
instructional design issues will be discussed. Solutions to difficult ToolBook issues will also be 
mentioned. 

Additionally, many individuals are familiar with HyperCard and the role it has played in providing 
the capability of combining text and graphics to create powerful applications without mastering a 
difficult programming language. There are numerous questions about the similarities of ToolBook 
and concerns about the transition from the HyperCard environment to the ToolBook environment. 
Questions about the similarities and differences of HyperCard and ToolBook and the move between 
these two environments will be addressed. 



IMPLEMENTING HYPERMEDIA CONCEPTS WITH HYPERTALK 



R. Scott Grabtnger, University of Colorado at Denver 

HyperCard® and HyperTalk® from Apple Computer are used frequently for the development of small 
scale hypermedia/hypertext systems. Hypermedia systems require complex development of links, 
nodes, maps, and networks of information from a variety of sources. The purpose of this session is to 
define some of the basic hypermedia concepts and illustrate how those concepts can be implemented 
with the HyperTalk language. 
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HYPERTEXT LEARNING ENVIRONMENTS AND COMPUTER-BASED 
DRILLS: A COMPARISON OF CONTRASTING DESIGNS FOR THE 
ACQUISITION OF COMPLEX KNOWLEDGE 



Michael J. Jacobson, University of Illinois 



Although the application of hypertext systems to educational situations has been attracting 
considerable attention recently, there has been little empirical validation as to the instructional 
effectiveness of such systems. A study was reported which employed two contrasting designs Tor 
structuring hypertext learning environments that were used to provide instruction in a complex and 
ill-structured content area, the social impact of technology. The experimental treatment incorporated 
several design features derived fwm recent cognitive instructional theory, in particular a procedure 
that used multiple presentations of case-based examples that highlighted different abstract facets of 
the domain knowledge. The comparison treatment consisted of a hybrid hypertext/computer-based 
drill design that recycled missed items using an increasing ratio review algorithm. 

The results of the study revealed that while the computer-based drill design led to higher performance 
on the measures of memory for factual knowledge, the more hypertext-like design promoted superior 
transfer to new situations. These findings suggest that hypertext learning environments which 
present the instructed knowledge by explicitly demonstrating critical interrelationships between 
case-specific and abstract knowledge components in multiple contexts would promote superior 
knowledge transfer. Some implications of these findings for cognitive instructional theory and for 
the design and use of hypertext learning environments and computer-based drills were also discussed. 



MEDICAL HYPERTEXT BASED UPON COGNITIVE FLEXIBILITY THEORY 



David H. Jonassen, University of Colorado at Denver 
Advanced Knowledge Acquisition 

In order to solve complex, domain- or context-dependent problems, learners must acquire advanced 
knowledge, which occurs after the introduction of knowledge and prior to the development of expertise 
(Spiro et al, 1988). Since introductory learning stresses simple reproduction of rigid examples based 
upon limited cases, learners often fail to achieve advanced knowledge. Without being able to reason 
with or apply the information that they acquire, learners will not be able to transfer the knowledge to 
novel situations. Since nearly every cases in medicine is unique, transfer skills are essential. 
Learners need instructional conditions that stress multiple interconnectedness and different 
perspectives on the same cases. Learners need flexible representations of domain knowledge. Hence, 
we recommend cognitive flexibility theory. 
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BRIDGING THE GAP BETWEEN INSTRUCTIONAL 
STRATEGIES AND COMPUTER COURSEWARE 



Annette Lamb, University of Toledo 

The development of effective, efficient, and appealing materials should be the goal of computer 
courseware designers. However with all the "bells and whistles" available in newer authoring tools, 
designers sometimes forget about basic design principles. Currently, much of the commercial software 
being developed is Ineffective. While some of the problems lie wfih incomplete or ineffective front-end 
analyses, Interactivity that is non-existent ("Press spacebar to continue") or at too low a level of 
processing are prevalent In other courseware materials. The nature of the information processing 
generated by courseware will determine what Is learned by the students. For novice designers one of 
the biggest obstacles to the design of effective educational software Is the translation of precise 
instructional strategies into computer courseware. This transfer from paper and pencil ideas to 
HyperCard cards and stacks can be mediated with some practical guidelines that relate directly to the 
events of instruction. 

The purpose of this presentation is to describe strategies for transforming instructional strategies 
into computer courseware materials. The session focuses on individual events of instruction and 
provides practical approaches for designing lesson elements. These Events of Instruction are: gaining 
and maintaining attention, informing the learner of the objective, recalling prerequisite skills, 
presenting the stimulus material, providing learning guidance, eliciting the response, providing 
feedback, assessing performance, and enhancing retention and transfer. Examples from HyperCard 
stacks will be provided for each of these events. 



MAKING GEOGRAPHY FUN WITH HYPERCARD 



Annette Lamb, University of Toledo 

This presentation will describe and highlight the development and use of two HyperCard -based 
geography activities. First, Banking Bucks in Brazil is a simulation that incorporates technology 
tools to support student participation In collaborative decision making, information exploration, 
and practice of higher level thinking skills. Middle school students participate in a collaborate 
approach to Information exploration through accessing, manipulating, and applying regional and 
investment information about Brazil. Students have access to regional and investment information, 
compare and contrast regions of Brazil, identify geographical information, evaluate possible 
investments, and select a successful investment. This project goes far beyond a simple simulation by 
providing access to a wealth of important information about the country, its regions, and its people 
through a hypermedia format. 

Second. Ralph's Retreats and Resorts is a HyperCard-based instructional game that helps students 
learn the terminology related to geography. Students must create a resort using a random set of 
geographic components. Information about the components is provided to assist users in Identifying 
and incorporating these elements into their resort. This game stresses the geographic themes of place 
and location. 

The presentation will describe the Integration of these activities and other simulations in a middle 
school social studies classroom. Finally, the presentation will highlight lesson extensions and other 
Ideas for HyperCard-based activities. 
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HYPERTEXT DELIVERY SYSTEMS 



Carol B. MacKnight, University of Massachusetts 
Santosh Balagopalan, Mentor Graphics 

Hypertext and Hypermedia have captured our Imagination with their capabilities to support 
instruction, learning, and collaborative research. For IBM users taking the first step often means 
selecting a hypertext system that will provide the links between text, graphics, audio, and video. In 
this paper, emphasis is on a comparison of two hypertext systems based on the object-oriented 
paradigm. 



INSTRUCTIONAL APPLICATIONS OP HYPERMEDIA: 
FUNCTIONAL* REQUIREMENTS AND RESEARCH ISSUES 



Ok choon Park, U.S. Army Research Institute 

The purposes of this presentation are: (a) to examine the functional features hypermedia systems need 
for instructional applications; (b) to describe important uses and limitations of hypermedia; and (c) to 
discuss research issues and technical improvements for hypermedia's instructional applications. 

Functional requirements of hypermedia depend on its use. First, this presentation will discuss the 
features and needs of the following functions: (a) guiding node selection; (b) opening multi windows; 
(c) changing window locations and sizes; (d) changing data base structure; (e) having browsers; (f) 
selecting nodes by key words; (g) having Independent storage Ales; (h) generating newly edited versions 
of the program; (i) Interfacing with hardware peripherals; 0) interfacing with programming languages. 

Second, values and limitations of hypermedia will be discussed for the following uses: (a) as an 
instructional delivery system; (b) as an idea generation and organization tool; (c) ai a relational 
information management system; and (d) as a CBI authoring system. 

Third, the following research issues will be discussed to clarify the instructional applications of 
hypermedia: (a) use of learner-control principles and strategies, (b) selection of information- 
representation forms, (c) use of a hypermedia-based knowledge base in CBI. and (d) development of 
intelligent hypermedia. 



DESIGN, DEVELOPMENT AND TESTING OP A 
CONTEXTUALIZED COMPUTER ENVIRONMENT 
FOR SPANISH LANGUAGE ACQUISITION 



Wayne A. Nelson, Southern Illinois University at Edwardsville 

Kathleen A. Bueno. St. Louis University 
Sajiah Md. Yusof, Southern Illinois University at Edwardsville 

Issues related to the design, development, and testing of a contextualized computer environment for 
Spanish language acquisition are discussed in relation to the role of computer in school settings and 
the need to embody cognitive learning theories in computer-based instructional systems. 
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INTELIGENT TUTORING WITH HYPERTALK: WHO NEEDS LISP OR PROLOG? 



Wayne A. Nelson,and Sqflah MdYusof, 
Southern Illinois University at Edwardsville 

Intelligent tutoring systems (ITS) represent a new approach to instructional software design that takes 
advantage of recent advances in cognitive science research. Rather than making all decisions about 
instruction before presenting the software to the learner, ITS utilize representations of learner and 
expert knowledge to diagnose learner errors and select instructional tactics and curriculum sequence 
in a dynamic process which unfolds as the learner interacts with the system. Such an approach 
allows the system to adapt to Individual performance in ways not possible for software developed 
using the traditional instructional systems approach to design. 

Object-oriented systems such as HyperCard provide a viable alternative for implementing 
instructional software which utilizes some ITS techniques. The presentation will discuss ways to 
Implement specific techniques for intelligent tutoring in HyperCard related to interface design, data 
structures for knowledge representation, diagnosis of errors, instructional transactions, and 
curriculum sequencing. A specific example of a system programmed entirely in HyperCard will be 
demonstrated. 



THE INSTRUCTIONAL DESIGN PROCESS: A HYPERCARD MODEL 



Lawrence C. Ragan, Perm. State University 

Faculty and staff interested in developing instructional courseware are faced with many challenges 
due to time constraints, Job requirements, and lack of direction or procedures to follow. Although 
motivation is initially high, interest soon lags as the "small" project dredges on for months or years. 
Most faculty or staff do not have formal training in the instructional systems approach and therefore 
work unfocused and misguided often without the program ever being completed. The Instructional 
Design Process (IDP) provides the developer with a model that illustrates the four major phases and 
accompanying steps in the development process. 

The IDP HyperCard stack was created as one method of guiding the potential developer through the 
relevant phases and steps in the instructional design model. This software was generated to provide a 
focus on such important details as learner characteristics, and task requirements. The developer can 
select and choose the appropriate steps for their development project and more realistically set goals 
and measure their completion. This stack has been used in Instructional Design Seminars with 
success at Penn State and in faculty-training programs in Costa Rica. 

The stack was not designed for advanced courseware developers and is not an instructional design 
management tool. 
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FULD INSTITUTE FOR TECHNOLOGY IN NURSING 
EDUCATION: "THERAPEUTIC COMMUNICATION" 



Barbara Breedon, Fuld Institute for Technology in Nursing Education 

The Fuld Institute for Technology in Nursing Education (F1TNE) is a non-profit corporation 
established by the Helene Fuld Health Trust to encourage and support the use of educational 
technology in nursing education Our second interactive videodisc production, 'Therapeutic 
Communication", is a multi-purpose educational tool for the analyf's and synthesis of 
communication strategies for nursing education. In the design of this interat itve videodisc program, 
there were some unique considerations made that may be of interest to other developers. 

As a project team, we endeavored to make the program friendly and accommodating to our target 
audience. Several factors were important: 

• the program should accomodate use by learners working in pairs, etc. 

• the program should easily adapt to group use 

• the program should support instructors in integrating it into their curriculum 

In teaching therapeutic communication, instructors have traditionally relied on role playing 
situations in the classroom. Recognizing this, we included Partner Exercises throughout the prc^ram 
that set up situations for the learners to role play with each other. This also accommodates the reality 
that quite often students will work in pairs at the systems and these exercises give them the 
opportunity to share, compare and discuss what they are learning. 

The disc was organized to facilitate the viewing of the character vignettes without the computer. In 
this mode, the videodisc is used like a videotape with more precise control. We provide, on the disc and 
in the teacher's guide, an index of frame numbers for the vignettes. The instructor can show the 
vignettes In the classroom on a large monitor or projection system and control the presentation with 
the remote. 

Along with the program package, we include a VHS Demonstration Tape that canfce kept in the LRC for 
students and faculty. It illustrates how to load the disc, bootup the system, enter the program and 
continue to move through the menus and chapters. The videotape format is more familiar to most 
learners and instructors and helps to introduce them to the interactive disc format. 

We also provide a very thorough Teacher's Guide wl f h th^ pngram. Besides describing the content and 
many features of the program, it also Includes a chs»w called "Suggestions for Integration" and an 
appendix with class materials to photocopy. 

This presentation will demonstrate "Therapeutic Communication" and how we accommodated our 
audience's needs within the program's design and the materials that accompany it. 
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A COMPUTER MANAGEMENT SYSTEM FOR PATIENT SIMULATIONS 



Lynn Johnson, The University of Iowa 

We are teaching clinical problem-solving skills, including diagnosis and management, to dental 
students using a series of interactive videodisc patient simulations. We call this series Oral Disease 
Simulations for Diagnosis and Management (ODSDM). (Finkelstein, et al.. 1988) ODSDM presents an 
oral pathology diagnostic schema and thirty patient simulations. ODSDM is a prototype for a 
comprehensive dental simulation project. This paper describes a computer management system 
which assigns student3 patient simulations, tracks their progress, and generates reports. 

Our computer management system was designed in response to the following needs. First, the 
sequence that students perform simulations is critical. Each student should be presented with the 
individualized sequence of simulations they require to master clinical decision-making skills. 
Second, maintaining records of completed simulations and student performance on each simulation 
is a time-consuming task for faculty. Third, the simulations and the simulation models require 
ongoing evaluation to ensure high quality instruction. 

The objective of the management system is to ensure that each student masters diagnosis. Mastery 
must be obtained at a specified level before advancing to the next level. The management system does 
this by individualizing the sequence of the simulations to adapt to the needs of each student. Students 
who learn diagnosis quickly need to complete fewer simulations than those students who learn 
diagnosis less quickly. 

The management system generates 200 demographic, progress, and system reports. 
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USE OP INTERACTIVE VIDEO IN PERFORMANCE 
APPRAISAL RATER TRAINING 



William A. Kealy, Texas A&M University 

A recent approach to the longstanding problem of how to Improve rating accuracy of performance 
appraisals has been to show idiosyncratic raters videotaped examples of specific levels of Job 
performance. This "frame-of-reference" approach was carried one step further. In this case, by 
developing an Interactive video program that lets users view a scene, rate the performance shown, and 
receive corrective feedback on their judgements. 

The Food Service profession served as the rated activity for the current project. Experts In the field 
Identified seven discrete categories of work performance, such as Menu Familiarity, while another 
group of experts generated, for each category, twelve written descriptions that represented the entire 
range of potential Job-related behaviors. These served as the "rough scripts" for producing 84 video 
segments, half depicting the Job performance of a waitress with the remainder showing the same 
behavior by a waiter. Each video scene was then rated by 24 former waiters and waitresses to 
determine its "true" score and confirm its equivalence to the original performance description. 

These video scenes formed the basis for an interactive vid(. ; rating exercise that was part of a 
comprehensive CBT lesson. This instruction, created through the SUPERPILOT authoring language, 
provided instruction on the importar ».e of accurate performance appraisal, typical rater errors (such 
as Halo ElTect). types of Job categories in the food service profession, and how to use the Behaviorally- 
Anchored Rating Scales (BARS) contained in an accompanying workbook. After this instruction, a 
Job category title appeared followed by a related video scene of a waiter's or waitress' performance. At 
the end of the scene, the viewer was asked to rate the behavior shown using the keyboard. For 
responses other than a true score, viewers received directional feedback for first attempts and 
motivational feedback when an error was made more than once for a given video segment. By 
contrast, accurate ratings yielded a statement on the favorable progress of the user who was then 
presented with another video scene to rate. 

This project is discussed in terms of development practices for interactive video with similar, 
attltudlnally-based outcomes as well as its potential application in research. 



INTERACTIVE VIDEODISC FOR NURSING EDUCATION: VITAL SIGNS 



Donna Larson, Grand Valley State University 
Barbara Breedon, Fuld Institute for Technology in Nursing Education 

Design, content, and production issues will be discussed during the demonstration of this interactive 
videodisc series currently under development at the Fuld Institute for Technology in Nursing 
Education. The content (the measurements and Interpretation of temperature, pulse, respiration, and 
blood pressure) reflects development of both psychomotor and cognitive skills in beginning 'level 
nursing students. 
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AN INTERACTIVE VIDEODISC APPLICATION ON SKILL ANALYSIS 
OF THE FREESTYLE AND BUTTERFLY IN SWIMMING 



Kirk E. Mathlas, Arkansas State University 
Eric E. Smith, University of Northern Colorado 

A call for interactive videodisc development by researchers In educational technology resulted In the 
production of this application. Discipline specific videodiscs have been needed In order to move away 
from the adaptation of interactive videodiscs for research. Having a greater quantity of topic specific 
videodiscs, according to educational technology researchers, would provide a larger arena of 
instructional topics to study. The purpose of this interactive videodisc application is to teach future 
teachers to analyze the freestyle and butterfly skills in swimming. Film segments which were 
combined with other video segments were borrowed from the American Swimming Coaches 
Association. The interactive application is designed to present correct and Incorrect freestyle and 
butterfly techniques. Questions are incorporated throughout the instruction. The final two modules 
present a random generation of video segments and questions which allow for analysis practice. The 
specific objective of this presentation is to introduce a new direction of interactive videodisc 
technology through the presentation of the application. 



AN INTERACTIVE VIDEODISC INSTRUCTIONAL DESIGN 
STUDY IN TEACHING SKILL ANALYSIS 



Kirk E. Mathlas. Arkansas State University 
Eric E. Smith, University of Northern Colorado 
Thomas M. Adams H Arkansas State University 

Researchers within the field of educational technology have called for discipline specific applications. 
Such applications would allow for Improved research in that they would increase control and allow 
for better identification of factors that influence the effectiveness of Interactive video instructional 
applications. Specific applications which best meet the capabilities present In interactive video still 
need to be studied. Therefore, the purpose of this study was to analyze Interactive video instructional 
design features relative to analyzing skills In swimming. Specific objectives of this presentation are 
to Introduce multiple regression results on six subjects who experienced an interactive video 
treatment in learning analysis of swimming techniques. Instructional design features to be presented 
include time of instruction, competency levels on specific modules, and repetition of modules relative 
to final gain scores. 
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INTERACTIVE VIDEO AS A RESEARCH TOOL IN EDUCATION 



Kirk E. Mathias, Arkansas State University 
Eric E. Smith, University of Northern Colorado 

Slee (1989) explained that media comparisons (i.e. interactive video vs. traditional teaching) with 
interactive video have been studied beyond the necessary limits and that an effort should be made to 
use IVD as a research tool. The purpose of this presentation is to present a position on specific models 
and learning strategies of research which can best be studied through the use of interactive video. This 
will include the following: recent evolution of technology specific to education, recent research 
relative to the sports arena, specific positions and ideas ofuse in the areas of research feedback on 
performance, teaching methods for preservice teachers, Gordon's cognitive model, interactive video 
instruction effects on sport skill performance and identification of cognitive processes in sport skill 
analysis. Research ideas for establishing identification of cognitive processes in pre-service teacher 
preparation will also be presented. 



AN INTERACTIVE VIDEO LESSON PLANNING 
PROJECTFOR TEACHER EDUCATION 



Dr. Charlotte Svherer, Bowling Green State University 

This teacher education project involves the development of a HyperCard stack to assist education 
students in planning for instruction. The stack is designed to control the MECC Improving Teacher 
Effectiveness laser disc, which calls up appropriate visual segments illustrating many of the effective 
teaching strategies. Although effective teaching strategies are stressed, the lesson plan process In this 
program does not adhere to a single approach to planning. Rather, the HyperCard stack is designed to 
facilitate general lesson planning while allowing for incorporation of effective teaching strategies. 

The materials were pilot tested in the spring semester of 1991 with elementary education students 
enrolled in Computer Utilization in the Classroom courses who needed to write lesson plans 
incorporating the computer as assignments for that class. In this project they used the IVD resource to 
analyze and apply principles that promote student learning. Research was completed to test for the 
effectiveness of this lesson plan writing strategy by comparing the quality of lesson plans written 
using this media with plans written in other ways. A questionnaire was also given regarding the use 
of the IVD media itself. 

Preliminary results indicate these materials are effective; however, the HyperCard stack alone may 
also be adequate. 
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BEHIND THE SCENES: THE MAKING OF A GENERIC RESEARCH VIDEODISC 



Eileen Schroeder, Marilyn Arnone, Richard Kenny, and 
Thomas Moats, Syracuse University 

Interactive video has been described as a tool for examining general questions i on ^^tional 
strategies messafie design, media/learner interactivity, and learner characteristics. It is also seen as a 
SRueXb^ worthy of examination. With the high cost and time requ red to 
produce a videodisc, the concept of a generic disc which provides a database of still and moving images 
and audio has arisen. A generic disc could be used to study a variety of different research questions and 
also be useful for developing a number of different lessons in real life settings. Its use in a variety of 
classes at different institutions could provide a large pool of research subjects. 

To develop such a generic research disc one must consider a number of questions dealing with the 
overall instructional design process, content/task analysis, selection and collection of images, on- 
dte^wsoff-dlae variables putting the images on a disc, budget, and a realistic timeline Decisions 
at each step of the way affect the utility of the disc for research and lesson development. Further, 
experience in the process of developing a generic videodisc has yielded lessons transferable across 
other new technologies such as DVI and CD-I. 

This presentation will examine the rationale for a generic research disc, advantages and 
disadvantages, development factors, and the tradeoffs of recent technological developments that can 
duplicate many of the features formerly available solely on videodiscs. 



NEW INTERACTIVE VIDEO DISPLAY CAPAIBILITIES: 
BASING DECISIONS ON DESIGN GOALS 



Dr. David Taylor, Virginia Tech 

The technical sophistication of new interactive technologies for displaying and merging video and 
computer graphics is increasing rapidly, and these capabilities can easily lead clients and developers 
astray when selecting a display system for their courseware. The basic choice is between a system that 
integrates computer graphics and text with video on one screen, versus simpler systems that place 
computer output (digital) and video (analog) on separate screens. The tendency is to choose the system 
that offers the most in terms of "bells and whistles", without adequately considering the implications 
for instructional design, project objectives, or the delivery environment. 

Two recent projects with different hardware and software platforms— a computer-based sophomore- 
level plant biology laboratory, and a tutorial/simulation for veterinary students— serve as examples 
of how decisions can be made in considering display needs and capabilities. 

This presentation includes: 

•An overview of available interactive hardware and software systems from the standpoint oi tneir 
dlsplay capabilities; ^ . 

•Theoretical factors (from research in perception and learning) related to instructional design and 
development for one or two-screen systems; 

•Practical reasons (e.g. , cost and space) for deciding between the two systems: 
•Guidelines for choosing a system, based upon project goals, instructional objectives and the user 
environment. 
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INTERACTIVE VIDEO AT THE MOTOROLA MUSEUM OP ELECTRONIVS 



Paul Wangemann, Motorola Inc. 

The Motorola Museum of Electronics offers a structured learning environment for corporate visitors 
and public school groups that emphasizes everyday cognition, authentic tasks, and real world 
problem solving. The museum combines the dynamic world of working professionals engaged In 
problem solving with the specific needs and challenges of the world In which we live In. 

The underlying assumption of the educational experiences available m the museum is that the 
learning process and environment are as important as the subject matter. The museum integrates 
multimedia Interactive video experiences into exhibits and provides extensive learning activities in 
computer and hands-on laboratories. In order for visitors to learn about the world of electronics they 
need to experience In a meaningful way. Computers play an Important role not only as a tool in the 
educational process but as a motivating example of how electronics helps to do Important things 
differently than in the past. 

The purpose of this presentation will be to showcase examples from the museum of multimedia 
interactives delivered via the computer. The interactives are integrated into a larger educational 
system or program. The purpose of the interactives is to provide motivating real world problem 
solving experiences in which visitors utilize electronics. 
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COST-OF-QUALITY PRINCIPLES RELATED TO COMPUTER-EASED TRAINING 



Robert Ccdlan , AT&T Corporate Education and Training 

This presentation will address the use of Cost-of-Quality (COQ) principles to identify areas for 
improvement in the development and delivery of CBT. The basic COQ categories will be described and 
illustrated with CBT examples. The COQ Identification Process will be explained and applied to the 
work done in a CBT delivery group. 



PLANNING FORMATIVE EVALUATION IN DESIGNING 
COMPUTER ASSISTED INSTRUCTION 



Lih-Juan ChanUn , The University of Georgia 
James R. Okey, The University of Georgia 

A formative evaluation plan for computer-assisted Instruction should plan for different assessments 
and use the acquired information to revise the product and processes to achieve the purposes of 
making learners learn better and the processes be conducted more effectively. Conducting the 
formative evaluation can be considered from three perspectives: assessing the achievement of the 
outcomes or objectives, assessing the effectiveness of the development process, and assessing the 
nature of the product. 

Traditional formative evaluation of CAI products has focused on assessing learner achievement of 
objectives and the inferential use of this information to make revisions. A more comprehensive 
approach to formative evaluation includes traditional learner achievement assessment but adds 
assessments made during the development process and of the product itself. Formative evaluation 
should begin well in advance of a trial product being ready for learner tryout. As soon as goals are set, 
instructional analyses carried out, or learner characteristics considered, formative evaluation of the 
decisions made and processes followed should be done by a development team throughout the 
development process. Additionally, the product should be examined to assure that it includes the 
characteristics and features consistent with learning theory and appropriate to the content being 
taught. a 

The three pronged approach to evaluation proposed here uses learners as a source of evaluation 
information but relies as well on content and instructional development experts. Together, the three 
perspectives provide a comprehensive approach to the development of effective and efficient CAI 
matt rials. 
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OUTGROWING THE COTTAGE: DEVELOPING 
COMPUTER-BASED TRAINING ON A LARGE SCALE 



Robert C. Frattni, AT&T National Product Training Center 

Developing, maintaining, and supporting two computer-based training courses is more than twice as 
difficult as doing the same for a single CBT product. Managing more and more CBT deliverables 
requires more than simply an increase In the scale of operations. There is a qualitative as well as a 
quantitative change In the nature of the tasks required. 

The Electronic Training Solutions (ETS) department of the AT&T Product Training Center has 
learned through the experience of such growing pains. By 1991, the department was responsible for 
the development, maintenance, and support of students using several hundred electronic training 
deliverables worldwide running on a range of delivery platforms. 

The department had recognized beginning In 1989 the fact that its organizational mission and role 
was changing. Internal processes and software tools were analyzed In terms of their ability to meet 
the needs of courseware production on the current scale, as well as organizational skills and the 
corporate product management environment. 

This presentation discusses the results of our analysis and provides a status report of our efforts. 
Software tools, development processes, and the role of CBT specialists have all changed in this new 
courseware management process. The techniques that we have used are appropriate for any electronic 
training development group which seeks to tailor their processes and tools to their specific 
organizational environment. 



THE MANAGEMENT CHALLENGE - CBT AT THE 
COMMONWEALTH BANK OP AUSTRALIA 



RJSpence 

The Commonwealth Bank of Australia is represented nationally across a network of 1500 branch 
offices. Training needs, coupled with distance education challenges, has seen the Installation of 1200 
stand-alone PCs dedicated to CBT. This session outlines the processes that were involved in 
establishing this large-scale CBT system and then concentrates on system acceptance, maintenance 
and the management of a courseware development group of 25. Present day challenges Include a 
change in the delivery platform operating system to OS/2. 
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THE USE OP INTERACTIVE VIDEODISC IN ACQUIRING LESSON 
PLANNING COMPETENCIES FOR MUSIC TEACHING 



Mary Leglar, University of Georgia 

The objective of the program is to introduce preservlce music teachers to techniques of instructional 
planning. The program addresses all aspects of planning for all types and levels of music classes, 
including performance-oriented classes, general music classes, and specialized classes such as theory 
and history. 

The first phase provides instruction in selecting and formulating objectives. In step 1 short video 
segments are used to illustrate computer-provided instruction. In step 2 the student practices 
identifying correctly stated objectives by viewing video segments and choosing behavioral statements 
that most accurately describe the lesson. In step 3, the student views a video segment and formulates 
an appropriate objective for that lesson. 

The second phase focuses on selecting and sequencing activities appropriate to a given objective. First 
the student watches a lesson segment and identifies and sequences the activities demonstrated. The 
student then views several classes using different strategies to teach the same objective, and selects the 
sequence of activities that most effectively accomplishes the objective. 

In the third phase, the student is introduced to various methods of evaluation. He then views selected 
lessons and chooses the best means of evaluation. 

The program is written for the Macintosh using HyperCard. 



ASSESSING MUSICAL ABILITY WITH COMPUTERIZED ADAPTIVE TESTS 



Walter P. Vtspoel The University of Iowa 

One of the most significant recent advances In educational and psychological measurement is the 
development of computerized adaptive tests. Research has shown that adaptive tests require from 50 
to 80% fewer items to yield reliabilities and validities comparable or superior to those of 
conventional tests (McBride & Martin. 1983; Moreno. Wetzel, McBride. & Weiss, 1984; Uny, 1977; 
Weiss. 1982). These promising results have led to the development of the first operational 
computerized adaptive tests, which presently include adaptive versions of (a) standardized aptitude 
tests, such as the Aimed Services Vocational Aptitude Battery (Sympson. Weiss. & Ree. 1982). and The 
Differential Aptitude Tests (McBride, 1986). (b) college placement tests (Stocking. 1987). and (c) 
achievement tests for elementary and high school students developed by the Portland Public Schools, 
the Montgomeiy County (MD) Public Schools, and the WICAT Systems Corporation (cf. Rudner. 1989). 

In prior ADCIS conferences (Vispoel. 1987; Vlspoel & Twing. 1990; Vlspoel. ColTman. & Scriven. 1990), 
the present investigator has described research about the first applications of computerized adaptive 
tests designed to assess music listening skills. Results from two recent computer simulation studies 
(Vispoel. 1990; Vispoel and Twing. 1990) have indicated that adaptive music tests require up to 67% 
fewer items to yield reliabilities and validities comparable or superior to those of conventional music 
tests. This research culminated in the development of MusiCAT. the first operational adaptive test of 
musical ability (Vlspoel. Coffman. & Scriven, 1990). The purpose of this session is to describe results 
from a "real data" study evaluating MusiCAT. Results from this study, consistent whh prior 
simulation studies indicated that MusiCAT required substantially fewer items to yield levels of 
reliability and validity comparable or superior to. those of commercially available music tests 
assessing the same skills. 




SELF-ADAPTED TESTING: A NEW METHOD 
FOR ASSESSING MUSICAL ABILITY 



Walter P. VispoeU The University of Iowa 

One of the most important recent milestones in educational and psychological measurement is the 
development of item response theory. Item response theory enables one to estimate examinees' ability 
levels and their corresponding reliabilities even when examinees respond to completely different 
items. This feature of item response theory makes " Computerized Self-Adapted Testing" --a new and 
Innovative assessment method-possible. In a self-adapted test, items are grouped into different 
difficulty categories, and examinees choose the difficulty level they desire before responding to each 
item. After answering each item, examinees are told whether they answered the item correctly or 
incorrectly. The test ends when an examinee's estimated ability level is measured at a pre-specifled 
level of reliability (,80, .85, .90, etc.) or when a pre-specifled number of items have been administered 
(15, 20, 25, etc.). Self-adapted testing appears to be particularly advantageous for test anxious 
individuals. In two recent studies (Rocklin & O' Donnell, 1987;1991). test anxious individuals 
achieved significantly higher vocabulary scores when taking self-adapted tests than when taking 
conventional fixed-item tests He., the same set of items is administered to all examinees) or adaptive 
tests (i.e., the computer administers items to examinees based on their responses to previous items). In 
this session, results from a study comparing self-adapted, adaptive, and fixed-item tests of musical 
ability will be described. 
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SHORTCUTS TO CREATING LEARNING MODULES 



Marianne Yoder, Northern Arizona University 

The purpose of this presentation is to demonstrate various ways to shorten the length of time 
necessary to write the PILOT code when developing learning modules. Templates can be created of 
design elements used often In each module. Assembling the templates when creating a new module can 
cut down on the amount of programming time. Considerations In designing the templates will be 
discussed, and demonstrated. Among the templates demonstrated will be: random selection of 
questions, learner calculator, return to previous screen, global window setting, and teacher's 
summary (a template for viewing and printing out how the learner did). A handout of the code 
necessary for these templates will be provided. 
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NEW FEATURES OF THE CYBIS SYSTEM 



Dean Christensen, Control Data Corporation 

Control Data is making significant changes to the CYBIS (formerly PLATO) system to enhance 
communications, support additional hardware devices, improved the user interface and take 
advantage of increasingly powerful workstations. An overview of the new CYBIS system will be 
presented and new features will be previewed. These features will Include: 

An Enhanced notes system, featuring 
Full color 

Graphical notesfiles 

80 column text mode option 

Notesfile compatibility with E-mail systems 

CYBIS Workstation Software, featuring 

Interactive Videodisc (IVD) support 

CD-ROM (Compact Disc-Read Only Memory) 

Windowing 

Pull-down menus 

Fine touch grid 

Screen captures and restores 

Image imports 

The new CYBIS (formerly PLATO) system will be presented along with the following new features: 

An Enhanced note » system, featuring 
Full color 

Graphical notesfiles 

80 column text mode option 

Notesfile compatibility with E-mail systems 

CYBIS Workstation Software, featuring 

Interactive Videodisc (IVD) support 

CD-ROM (Compact Disc-Read Only Memory) 

Windowing 

Pull-down menus 

Fine touch grid 

Screen captures and restores 

Image imports 
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COURSEWARE EVALUATION 



C. B. Dunnagan, Control Data Corporation 

This presentation addresses one way that developers of computer-based instruction may have their 
courses evaluated and recommended for college credit. In 1979 Control Data had several of its courses 
evaluated and recommended by the American Council on Education (ACE). The number of courses has 
expanded and now numbers more than 50. The ACE evaluators are subject-matter specialists in the 
field under consideration. Evaluators are solicited from postsecondary educational institutions, 
professional and educational associations, and accrediting agencies recognized by the Council on 
Postsecondary Education. The review team may include one practitioner from a noncollegit ? 
organization. 

The presentation further discusses the benefits such a course evaluation has for potential students, 
present and future employees in the knowledge-based/information age, and colleges entering the 
distance learning field. 



AUTHORING UTILITIES THE HIDDEN POTENTIAL OP PLATO 



Robert Elmore, Control Data Corp. 

For more than two decades PLATO (now reborn as CYBIS) has been known as a comprehensive 
instructional system, combining quality interaction and graphics, a huge library of courseware and 
possibly the best user communications system available anywhere. What it has lacked is an easy way 
to author courseware. 

As part of the new suite of features being made available on CYBIS Control Data is including several 
authoring utilities. This presentation will outline thi«? "toolbox" concept of an authoring 
environment, where a variety of authoring tools, from models to authoring systems to authoring 
languages, is made available to fit the needs of different authors and different applications. It will 
also briefly introduce each authoring tool with some examples of courseware developed using these 
tools. 

The authoring tools will consist of the following: 
Authoring models 

The authoring models will include tutorial, simulation, case study, drill/practice and testing models. 
Authoring Systems 

Two or more CYBIS-based authoring systems will also be presented. These authoring systems will 
allow authors more flexibility than the authoring models, but will not require programming skills. 
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REPACKAGING: ONE KEY TO COST-EFFECTIVE TRAINING 



Elizabeth Miller, Control Data Corporation 
Cheryl Shankman, Hughes Training Systems 

Frequently companies reject "canned" training materials with the cry, "That's nice, but that's not what 
we do!" Upon closer inspection many of these programs contain 80 - 90 percent of the necessary 
training, but are missing the 10 • 20 percent that customizes it for a company or department's specific 
needs. Using real-life examples from DoD, industry, and manufacturing, the presenters will discuss 
their experience with "repackaging" existing materials and/or developing new materials to provide 
cost-effective training solutions. 



USING A WORD PROCESSOR TO DEVELOP 
COMPUTER-BASED INSTRUCTION 



Kevin Stall 

Many developers and designers neglect one of the most obvious tools that make a programmers life 
easier. Using a word processor can Increase not only the developers and designers productivity, but 
also the programmers. By using macros and other techniques the programmer will be able to do all 
the programming utilizing the word processor. This is particularly true with Plato. 
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THE EFFECTS OF VARIED LEVELS OF INTERACTIVITY 
IN IV ON LEARNING AND SOCIAL PROCESSES 



Margaret L. Bailey, Arthur Anderson & Company 
Dr. Jack Byars, Kansas State University 

Much of the research to date In interactive video has focused on comparing IV to CAI or stand-alone 
video. Few studies have attempted to define and vary the unique attributes of interactive video and 
systematically compare the effects on learning and the social environment. This presentation will 
discuss a research project which investigated the effects of varied levels of interactivity (defined as the 
number of opportunities for computer-user interaction) in an interactive video physics lesson on 
learning and social processes. 

Students enrolled in a college-level physics course were paired and assigned tc one of three 
interactivity levels (No, Low, High) of an interactive video lesson. Data was gathered and comparisons 
made on students' ability to retain and transfer lesson concepts, as well as attitudes toward the 
learning environment. In addition, a detailed analysis of the peer interaction patterns, as obtained 
through audio-tape recordings, was completed. 

Preliminary results indicate that while High levels of interactivity may result in significantly more 
positive attitudes, Low levels of interactivity result in greater retention of lesson concepts. Transfer 
effects, in this study, while somewhat greater for the High interactivity treatment, were not 
significantly greater than No or Low interactivity levels. 

In addition, it was found that social interaction was progressively more frequent as interactivity 
levels Increased. However, detailed analyses reveals that a greater percentage of the interaction 
among peers in the High interactivity treatment can be classified as procedure-related (reading aloud, 
asking for suggestions as to what to do next) rather than task-related (asking/answering content 
questions, giving explanations). 

This presentation will conclude with a discussion of the implications of this research and suggest 
future areas of study in Interactive video software design. 
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COMMUNICATION STRUCTURE ON AN 
ELECTRONIC TEACHERS' NETWORK 



John R. Broholm, University of Kansas 

This study examined the communication patterns and relationships among teachers using an 
electronic mail (E-mail) and shared database system. The system of interest. UNITE, was designed and 
implemented specifically for primary and secondary schools, including users and resources in all 
teaching content areas. The effects of teaching content area, school level, and participation in a 
collaborative teaching project were studied as communication constraints on UNITE. 

Schools have been characterized as work environments where teachers \» rk in isolation from each 
other. Yet teachers maintain a norm of interactive behavior that puts some emphasis on collegiality - 
- their willingness to be helpful and cordial to one another balanced against autonomy in determining 
exactly how they will teach. A new medium, computer-mediated communication (CMC), Is being 
introduced into schools, and may have profound effects on how teachers communicate with each 
other. 

A network analysis revealed that users of the UNITE electronic mail system seemed to overcome the 
constraint of spatial distance between communicators quite effectively. However, being in the same 
teaching content area was a significant structural parameter of communication -- possibly the most 
powerful parameter extant. School level was also a significant constalnt, but participation In a 
collaborative teaching effort did not prove significant. 

A clique-detection program, NEGOPY, found two clusters of communicators on UNITE, one larger and 
relatively heterogeneous, the other smaller and largely made up of librarians. Science teachers were 
the largest group using the system, but librarians used it more heavily than other teachers, mostly to 
communicate with librarians in other schools, which was an extension of their communication 
routines when not using CMC. As with most new technological systems, a few individuals accounted 
for a disproportionate amount of use. 
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THE EFFECTS OF COMPUTER SELF-EFFICACY ON DIRECTION-FOLLOWING 
BEHAVIOR IN COMPUTER ASSISTED INSTRUCTION 



Randal D. Carlson, The Pennsylvania State University 
Barbara L. GrabowskU The Pennsylvania State University 

In many cases, instruction requires that students precisely follow directions provided to them. This 
may be especially important in the case of computer-aided instruction (CAI) where the individual 
navigates through the instruction independently. In this case, cavalier attitudes toward following 
directions may result in the learner becoming "stuck" or confused. In spite of the importance of this 
instructional design issue, there has been little prior research which examines various patterns of 
direction-following behavior. One important aspect of this issue is understanding what types of 
individuals attend to and follow directions. 

In this study, a CAI lesson provided the framework to study the relationship between direction- 
following behavior and computer self-efficacy, participation in military training programs, and 
gender. A sample of 57 undergraduate students completed a module designed to provide them with the 
computing skills necessary to perform elementary operations on the mainframe operating system. 
VM/CMS. Students' direction-following behavior was measured by counting the number of directions 
in the CAI program that were properly followed. Their confidence in their ability to use a computer 
was measured by a Computer Self-Efficacy scale developed by Murphy. Coover. and Owen (1988) 
Analysis of the results using ANCOVA to control for time indicated a main effect for ROTC 
membership and a disordinal interaction between gender and self-efficacy in relation to direction 
following behavior. In addition, the covariate. time to complete the instructional module, was found 
to be significantly related to direction-following behavior. Reasons for the discovered relationships, 
implications for instructional design, and suggestions for future research are presented 
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CENTER FOR INNOVATIVE INSTRUCTIONAL TECHNOLOGY 



Stiie Farrell PhD, U S. Army Construction Engineering Research Laboratory 

1. How do a University and a Department of Defense Research Laboratory collaborate to serve each 
population in attaining objectives of a "Center for Innovative Instructional Technology"? 

2. Can criteria be developed by which problems can be classified to ease the task of matching 
situation/technology and anticipated audience? 

3. Can the Center's area of expertise be usefully expanded with the Army problems and the Research 
Laboratory open this unique service to other DoD problems. These are a few of the probing 
questions that are the basis of a research project currently underway at U. S. Army Construction 
Engineering Research Laboratory (USACERL) and the University of Illinois. The University 
provides a rich and deeply committed foundation in Instructional Technology. USACERL is replete 
with examples of unsolved or poorly solved problems that can best be addressed by Instructional 
Technology. Interagency cooperation and the fostering of "mentoring" or leadership roles in any 
pursuit of learning are encouraging the mutual relationship put forth in this research. 

Discussion will focus on the process of establishing and nurturing a center. It will present results and 
recommendations from other related research, current objectives and measures, as well as future 
goals and applications. Implications and recommendations for future research will also be 
presented. 



VIEWER PERCEPTIONS: MODEL SCREENS VS. ACTUAL SCREENS 



R. Scott Grabinger, University of Colorado at Denver 

This session reports results from a study which examined viewer perceptions of a variety of display 
screens to identify constructs to guide the design of those screens. Specifically, the study examined 
the effects of CRT screen models vs. actual screens from CAI programs on viewer preference and the 
relationships among viewer preference and viewer field articulation (field dependence/independence) 
and conceptual style (relational/analytical). 
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A PROPOSED TAXONOMY FOR THE DESIGN, STUDY, AND 
EVALUATION OF INSTRUCTIONAL SYSTEMS 



Dale Y. Hoskisson, Valley City State University 

Researchers are constantly comparing one Instructional system with another. Computer-based 
systems are being compared with "traditional systems." But these studies often provide no help In 
improving instruction because they do not tell us much about the system itself. We need to have a way 
to describe an instructional system Independent of the medium used or the theory espoused. This 
presentation explains a proposed taxonomy for Instructional systems that would allow the researcher 
and the consumer to evaluate a system Independent of medium or theory. This approach describes the 
ability of a system to manipulate information to facilitate learning. Such a taxonomy will make 
comparing systems a more useful and viable activity for research. The language consists of: 

• an objective classification and presentation description element based on Merrill's 
Component Display Theory 

• a measurement of interactivity from static to dynamic 

• terminology to describe interactivity addressing su^h items as number of facts (facts is used in 
a very general way here) presented, method of connecting facts, time allowed to connect facts, pacing, 
and information for problem solution 

• a presentation mode component— discovery, expository, or combination 

• a complexity component— simple to complex. 

The presentation will include the results of applying the language to four representative instructional 
systems which will include print-based and level three interactive videodisc systems. 



THE FORMATION AND IMPACT OF USERS' MENTAL MODELS 
IN HUMAN-COMPUTER INTERACTION 



Hueyching Janice Jih and Thomas C. Reeves 
The University of Georgia 

Computer users struggle to understand the structt re and functions provided by various programs and 
to learn the correct procedures of operation for programs that are supposed to help them accomplish 
tasks. Often a program is designed form the designer's points of view and not from the perspectives of 
users. As a result, users end up lacking the clear conceptual understanding about the software's 
structure and functions that the designers have. This problem can be especially acute when users are 
given the freedom to explore hypertext materials in which the possibility of user disorientation is a 
major concern. Currently, there is a growing concern about exactly what users know about computer 
programs In terms of users' mental models. 

Mental model theory is an attempt to model and explain human understanding of objects and 
phenomenon during learning, in problem solving, and when we are reflecting on or attempting to 
rationalize or explain the system's behavior. Research on mental models in human-computer 
interaction can identify salient characteristics of cognitive processes and help in the development of 
research-based guidelines for the design of effective courseware. 

What is a user's mental model of a computer program? What is the distinction between a computer 
program, the designer s conceptual model of the program, and the user's mental model of the program? 
How does a user develop his/her mental model of a specific program? What is the role of users' mental 
models In human-computer interaction? How could the mental model aspect influence the design 
consideration of computer programs? These are the central issues that this article will discuss. 
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COGNITIVE OUTCOMES OF BUILDING EXPERT SYSTEMS 



David H. Jonassen 
University of Colorado 

Researchers and businesses are Investigating the effects of having students and employees construct 
expert system rule bates. What Is Important are the cognitive effects of the construction process more 
than the resulting advisors. Expert systems can be thought of as cognitive tools engaging learners In 
cognitive and metacognitive learning outcomes. This project proposes to investigate the changes In 
critical thinking skills, cognitive learning strategies, and metacognitive skills that result from 
employees and students constructing rule bases. Changes in Individuals' knowledge structures will 
also be Investigated. 



SECOND GENERATION INSTRUCTIONAL DESIGN 



Mark Jones and Dave Merrill 
Utah State University 

The overall goal of the Second Generation Instructional Design (ID2) Research Program Is to develop a 
comprehensive new generation of instructional design theory and methodology, and to build a set of 
Intelligent tools that aid the designer In applying the theory. There are a total of six funded research 
projects currently active in the program. 

Dave Merrill will summarize the latest theoretical advances that have resulted from the research, and 
Mark Jones will demonstrate the most recent software developments. 
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THE GENERATIVE EFFECTS OF GRAPHIC ORGANIZERS WITH 
COMPUTER-BASED INTERACTIVE VIDEO 



Richard F. Kenny, Syracuse University 
Barbara L. GrabowskU Pennsylvania State University 

This presentation reports on the results of a study of the use of graphic organizers In conjunction with 
an interactive video on Nursing practice. Twenty nine university Nursing Juniors were asked to 
complete three of four modules of a CBIV program entitled Nursing an Elderly Patient with Chronic 
Obstructive Pulmonary Disease. With each module, participants were presented one of two forms of 
pictorial graphic organizer: Final Form Graphic Organizer (filled in) or Participatory Graphic 
Organizer (with various blanks to be filled to). Participants completed two posttests: immediately 
after the third module and one week later. 

The purpose was to investigate the differential effect of the organizers on immediate and delayed 
learning according to Wittrock's Generative Learning Hypothesis; that is, comparing the amount of 
learning from the final form graphic organizer to that from the more generative participatory graphic 
organizer. Further, the investigation examined the interaction between the graphic organizer 
treatments and the cognitive processing style of students who are analytic (those who have the ability 
to abstract a common dimension across stimuli) or holistic (those who have the ability to construct a 
whole from information about its parts). 

It was hypothesized that the more generative (Participatory Graphic Organizer) treatment group would 
achieve a significantly higher mean score on both Posttest 1 and Posttest 2. As well, it was postulated 
that the less generative (Final Form Graphic Organizer) treatment group would display a significantly 
greater mean loss score (from Posttest 1 to Posttest 2). Results did not support these hypotheses but 
provided some evidence in favor of advance organizer theory. Note -taking behavior on the part of 
most participants may have provided a substantial confounding factor. Neither analytic nor holistic 
ability appeared to interact with the treatments. These results will be discussed as well as 
implications for instructional design and further research. 



AN INTERACTIVE VIDEODISC INSTRUCTIONAL 
DESIGN STUDY IN TEACHING SKILL ANALYSIS 



Kirk E. Mathias, Arkansas State University 
Eric E. Smith, University of Northern Colorado 
Thomas M. Adams II, Arkansas Stair University 

Researchers within the field of educational technology have called for discipline specific applications. 
Such applications would allow for improved research to that they would increase control and allow 
for better identification of factors that influence the effectiveness of Interactive video instructional 
applications. Specific applications which best meet the capabilities present to interactive video still 
need to be studied. Therefore, the purpose of this study was to analyze interactive video instructional 
design features relative to analyzing skills to swimming. Specific objectives of this presentation are 
to Introduce multiple regression results on six subjects who experienced an interactive video 
treatment to learning analysis of swimming techniques. Instructional design features to be presented 
include time of instruction, competency levels on specific modules, and repetition of modules relative 
to final gain scores. 
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THE EFFECTS OP STRUCTURE AIDS ON LEVEL OF LEARNING AND 
EFFICIENCY IN AN INSTRUCTIONAL HYPERMEDIA ENVIRONMENT 



Timothy L. Phillips, The University of Iowa 
Ginger Watson, The University of loua 
Jay Cook, The University of Iowa 

Hypermedia allows learners to customize the Instruction to meet their needs. However, in 
implementing hypermedia-based instruction the effects of the following factors should be considered: 
learner disorientation, cognitive load, and learner control. Hypermedia-based instructional 
environments require learners to make numerous branching decisions. Research has indicated that 
learners are not always capable of assessing their Instructional needs. Secondly, learners must keep 
track of their current location and the information path taken, In order to not become disoriented. 
Many learners find the demands too great and become disoriented. The combined cognitive load of 
learner control and learner orientation may result in reduced comprehension. 

One possible way of reducing the cognitive load of a hypermedia environment is the use of structural 
aids. Structural aids (outlines, indexes) have been found to aid learners in organizing and 
comprehending information. Different types of structural aids have been found to effect both the 
quantity and level of learning. It has been suggested that providing learners with a structural aid 
would reduce the cognitive load on the learner resulting in increased comprehension. Therefore, the 
purpose of this study is to evaluate the effectiveness of Structural Aids (hierarchical index, 
alphabetical Index) on Level of Learning (Fact, Inference) in a hypermedia lesson. 



THE EFFECTS OF GENERATIVE CONCEPT-MAP STRATEGY IN CBI 



Lawrence C. Ragan, Perm State University 

The generative learning theory proposed by Wittrock and others describes meaningful learning 
occurring when the learner generates, or constructs new meaning from presented stimuli, and existing 
knowledge and experiences. Incorporating generative learning strategies into CBI presents a unique 
set of challenges to todays courseware developers. Generative strategies embedded in the software may 
elicit specific learning outcomes. This presentation wlU present the results of research on the effects 
of one generative learning strategy, the concept map technique, on comprehension scores measured an 
Instrument designed after Royer's Sentence Verification Test. The research hypothesis Is directed at 
examining the effects of generating concept-nodes and relationship-links on comprehension of 
textual passages about a fictitious country in Africa. 

Based on schemata network theory and spatial learning strategies by Holly and Dansereau, a 
HyperCard-based technique was developed that enables the learner to generate concepts-nodes and the 
relationships-links between those concepts after the presentation of information. The two 
treatments in this study consisted of one group that constructed a concept map representation of the 
the textual passage, and one group receiving system supplied concept map representations of the 
textual passage. This seminar will demonstrate the generative concept -map HyperCard stack as well 
as discuss results from the pilot study conducted in the spring of 1991. Plans for future research based 
on the pilot study will be explained as well. 



DEVELOPMENT OF AN INTELLIGENT TUTORING SYSTEM IN ACCOUNTING 



Thomas H . Rowley, Georgia State University 
Michael A. Orey, The University of Georgia 

It Is difficult to develop interesting and highly interactive computer-based instruction (CBI) in 
accounting. The reason for this difficulty include the fact that accounting is a rule-based discipline, 
and accounting learning is perceived to be linear. That is, the conventional teaching model is to build 
a student's knowledge base Incrementally along a carefully developed path. Conventional teaching 
strategies require a structure (syllabus) that outlines the expected procedural growth of a student's 
knowledge-base. Use of conventional teaching models to develop CBI often result in 'page turning' 
tutorials. 

Faculty who are seeking technologically sound, interactive vehicles for accounting education will be 
interested in the recent developments fo educational technology. Intelligent Tutoring Systems (ITS) 
may be one such development that may impact on accounting education. This type of technology 
permits learners to work on problems while a "smart" tutor monitors their progress. In this way, 
fundamental misconceptions can be eliminated before they become so entrenched that it would be 
difficult to correct that misconception. 

This presentation describes the conceptual development of an ITS to provide accounting students with 
assistance In learning the concepts and principles of preparing a statement of cash flows. The 
presentation will first summarize the background literature from the perspective of prior CBI efforts 
and identifies the success factors for development of ITS. The second section describes the design 
process for an ITS application In accounting. 



EXAMINING EDUCATIONAL SOFTWARE FROM A CULTURAL PERSPECTIVE 



Penelope Sernrau, Ph.D„ California State University, Los Angeles 

Research has demonstrated that different cultural groups perceive and decode visual materials in 
different ways (Heinlch et ai, 1982, p. 67). Yet, Digranes and Digranes' research (1989) found that 
instructional materials specific for various cultural groups was virtually non-existent. 

How are women portrayed In the images seen In educational software products? Are women shown in 
successful careers? How about men? Are men shown as nurturing, caring beings? Are there just as 
many images of women as men; or, is there a greater volume of male Images? Are the elderly and the 
physically impaired represented equally with images of the young and physically able? What cultural 
values are being projected through the software regarding Latinos, Asians, and Afro-Americans? 

How does one analyze the visual images in educational software from a cultural perspective? In this 
presentation guidelines for examining images from software products such as Reader Rabbit and 
Carmen San Diego will be discussed. The speaker concludes that certain values and attitudes are being 
projected and learnt by children while using the soffware. When educators evaluate software, they 
need to be concerned with the underlying cultural values being taught. 
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ADAPTIVE ON-LINE HELP FOR CONCEPT DEVELOPMENT 



Doris S. Shaw, L. Michael Golish, 
Julie L. Webster, and Der-Shung Yang, 
U.S. Army Construction Engineering Research Lab 

Our previous research found that embedded Instructional systems were effective for teaching 
computer-aided design (CAD) to architects, engineers, and technicians. Also, understanding CAD 
concepts, how the system functioned, was instrumental in Increasing the design effectiveness of users, 
and experience in similar systems was a significant factor in learning those concepts (Shaw, 1988). 

Recent research examined the effectiveness of a different approach to teaching CAD concepts. On-line 
help aimed at presenting conceptual information was designed to supplement the procedural help that 
had been available In our previous instruction. The program was set to direct users with some 
experience to the conceptual help screens when they requested "help". Asking for "more help" showed 
them the procedural help. When the learner indicated that he/she was not experienced, the procedural 
screen appeared first, but the conceptual help could be seen by requesting "more help." The system was 
implemented with an adaptive feature that would exchange the initial help screen setting if the user 
continuously requested "more help." 

Fifty-four adult professionals responded to a survey that indicated the learner's satisfaction with the 
help system. Novice users who consulted the help screens profited greatly from the help and felt 
confident that they could use the system. Learning style and previous conceptual understanding 
influenced the other three groups. Some learners experienced in other systems reported a high level of 
understanding without the use of help, but those who requested help did not always profit from it. 
Novices who did not select help often found the lessons difficult. 



DESIGNING MICROCOMPUTER-BASED INSTRUCTIONAL 
SIMULATIONS: IMPLICATIONS FROM COGNITIVE PSYCHOLOGY 



Richard A. Thurman and Joseph S. Mattoon, 
Air Force, Armstrong Laboratory 

The entrance of the microcomputer into the educational system resulted in an Increased Interest in 
the design and use of computer-based Instructional simulations. This type of simulation has the 
potential to enhance the transfer of learning by teaching complex mental and procedural tasks in 
environments that approximate fairly realistic settings. However, many of the microcomputer-based 
instructional simulations produced today are poorly designed from a psychological standpoint and 
therefore do not provide an effective learning environment. This presentation represents our attempt 
at identifying areas in the psychological literature which have important implications for the design 
and use of Instructional simulations. In this presentation we focus on five major issues: (a) cognitive 
structure, (b) cognitive strategies, (c) metacognition, (d) automaticity, and (e) affect. Each issue will be 
explored by summarizing representative research and findings, and by presenting implications for 
the design and use of instructional simulators. 



ERIC 



CUT APPLICATIONS OP KOLB'S EXPERIEMENTIAL LEARNING THEORY 



Michael Yacci, Rochester Institute of Technology 

Kolb's Experiential Learning theory is a descriptive theory about how people learn. Briefly, it suggests 
that learners can be divided into four quadrants based upon the way they process information. 
However, it can be used as the basis for a prescriptive model of Instructional design, by creating 
instruction that sequentially moves through each quadrant, therefore appealing to a learner's 
strengths while simultaneously challenging the learner to process information in new ways. 

This theory can be used in the design of CBT and Interactive Video as a guideline for engaging the 
learner and taking him or her through the four learning quadrants. This presentation suggests that 
there are techniques and methods of CBT design that correlate with the cognitive processes that occur 
in each quadrant, and that a CBT lesson based on Experiential Learning can use these methods in a 
deliberate manner. Experiential Learning suggests a prescriptive sequence for common CBT strategies 
and techniques, such as online notepads, hypertext, tutorials, and simulations. 
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THE UTRECHT TELECAI-PROJECT 



Drs. J.P. Bakker, Hogeschool Utrecht (Utrecht Polytechnic) 

In the Utrecht TeleCAI-proJect 20 students from Utrecht Polytechnic study at home using courseware 
and a telelink with the teacher. The telelink is used for transfer of logged data from the CAI-lessons 
and for electronic mail between students and teacher. 

Main objectives of the project are to decrease time and place constraints for the students (flexibile 
learning), to increase personal teacher support and to decrease traffic at rush-hour. 

Results of the project after one year are that the e-mailsystem served well for exchange of questions, 
for delivery of extra exercises and for exchange of information about study-progress. The use of e-mail 
for personal messages was less than expected. Direct interpersonal contacts were preferred. The logged 
data was a good predictor of study-success. Students appreciated very much the flexibility of the 
learning process, but found the (standard) e-mail service still too cumbersome to use. 

The end-conclusion was that the introduction of telematics as a learning tool shifts the roles of 
teacher and students. It takes time to find new ways for both. The e-mail system must be more user- 
friendly and specifically dedicated to the learning task. 

The project is a unique experiment in the Netherlands and will continue on a larger scale in 1991/92. 



COMPUTER COMMUNICATION-FOUR INNOVATICE 
PROJECTS AT PENN STATE UNIVERSITY 



Barbara Grabowski, Perm State Unive. stty 

The purpose of this symposium is to describe four projects being Implemented at Penn State 
University. The symposium will begin with an overview of Innovative Instructional uses of electronic 
communication as demonstrated by an advisory tool developed using HyperCard. The presentations 
will consist of : 

INSTRUCTIONAL USES FOR ELECTRONIC COMMUNICATION: AN ADVISORY TOOL FOR NOVICE 
FACULTY USERS 

PROBLEM SOLVING AND LOGICAL THINKING with 

-CLASSNEWS" AN ON-LINE COMPUTER DIALOGING FACILITY FOR THE CLASS, 

ENHANCING COMMUNICATION WITH A COMPUTER CONFERENCING SYSTEM AND DEOS - THE 
DISTANCE EDUCATION ON-LINE SYMPOSIUM 

TEACHER COLLABORATION THROUGH REGIONAL AND STATEWIDE BULLETIN BOARD SYSTEM 
ELECTRONIC COMMUNICATION: A SYNTHESIS AND CRITIQUE. 

The presentations will describe three different uses of the medium and present implementation 
results. Finally, all of the presentations will be synthesized and critiqued by the discussant. 



ERIC 



85 

1C1 



SCIENCE INSTRUCTION ON GLOBAL ELECTRONIC NETWORKS 



Michael L. Waugh, University of Illinois at UrbanaChampaign 
James A. Levin, University of Illinois at Urbana Champaign 

Real world science Is conducted In global contexts, with collaboration among colleagues around the 
world a common occurrence. With the spread of microcomputers Into science classrooms and the 
availability of electronic networks, it is now possible to conduct science instruction in such global 
contexts as well. This presentation will discuss ongoing research and development efforts concerning 
ways in which electronic networks can be used In science teaching and learning. Electronic networks 
can allow students to participate In distributed science problem solving activities by providing 
"dynamic support" so that students can gather and analyze data and Jointly create project reports with 
students from other parts of the world. Several of these "telescience" activities will be described in 
detail In one activity, students from across the United States have been engaged In data collection 
and data analysis activities related to estimating the circumference of the earth. In another activity, 
students and teachers participated in the collection and analysis of data concerning the cost, 
nutritional value and appeal of typical school lunches. The session will conclude with a discussion of 
the potential for electronic networks to improve science teaching and learning. 
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COMPUTER AIDED TESTING MAKES STUDENTS WORK REGULARLY 
(AND INCREASES TRANSFER RATES DRASTICALLY) 



M.W.J.W. Dykman, Delft University of Technology 
R.HJV. Stoat, Deljl University of Technology 



Abstract 

First year university science courses often have 
an hierarchical content. Hence It Is very 
Important that students work regularly. 
Weekly computer aided testing stimulates 
students to follow a regular pattern of studying. 
This gives them a great help and shows clear 
increases in the transfer rates of 3 mechanics 
courses in the first and second year. 

Introduction / educational context. 

In the last 25 years the developments within 
many Dutch universities were strongly 
determined by a large increase in student 
population and at the same time a constant or 
decreasing number of lecturers available, 
especially In the first 2 years of the study. 

At the Technical University of Delft the student 
population in several faculties was roughly 
doubled from about 150 to over 300 per year. 
Because of the lack of good trained teaching 
staff many small classroom lectures for about 
30 students (tutorials) were stopped and the 
only way to give the student oral Instruction 
was to give lectures in the large auditorium 
with one lecturer for 300-400 students. 

This was especially the case for the courses in 
mechanics in the first and second y r. The 
idea was that by means of good books for 
selfstudy and other written teaching facilities 
students hopefully would have the same results 
as In the former situation with more teaching 
and exercise facilities in small classrooms. 
Moreover In that time many people Inside and 
outside the universities clung to the theory (or 
belief) that the students had to be treated as 
complete Independent creatures who could pave 
their own way through the Jungle of the 
university courses and exams. 

So the tendency was that teachers were not for 
teaching, but for producing textbooks, other 
written materials and exams. Besides that the 
system of examination changed also In those 
years. It became a very free system; every exam 
could be done as many times as the student 
wished. 



This however turned out not to be a good 
approach to education at a technical 
university, at least not for the courses like 
mechanics. Although in many occasions the 
results of the exams were not that bad (around 
50 %), the transfer rate of the students through 
the curriculum decreased dramatically. Many 
students postponed the exams untill the next 
year(s), because they started too late doing 
exercises by themselves. 

In 1988 the faculty of Civil Engineering 
switched from a system of many parallel 
courses, In which each course is taking several 
months to over half a year, to a modular 
system of short courses, which each take 4 or 8 
weeks, with only 2 courses running at the same 
time. 

So the lecturers of mechanics In the faculty of 
Civil Engineering were placed for the serious 
problem: "How to get students working 
immediately, without seeing or knowing them 
personally". 

In that situation the call for Computer Aided 
Instruction was obvious. From the beginning 
the starting principle of the task of the 
computer was not to teach or instruct the 
student difficult parts of the courses, but it had 
to serve as a kind of testing machine for the 
student to see for himself whether he is 
studying in the right way and to give him a 
bonus for the coming exam. It must motivate 
the student to start working in time at home. 
For that purpose in the years '89-'90 an 
interactive testing system has been developed, 
which is used for several courses In mechanics 
in the first and second year of the faculty of 
Civil Engineering. These are Mechanics I (MI). 
Fluid Mechanics (FM) and Mechanics II (M I). 
MI is taught in the first year and FM and Mil in 
the second year. 

The description of the system and the effect on 
the transfer rate of the students through the 
courses are described below. 

Organization of the courses 

We will describe the organization of the MI 
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course lii more detail. The organization of the 
other courses Is equivalent. 

Mechanics I Is a basic subject which is taught 
in many technical and science curricula. In our 
umvei *.ty the average student spends 80 hours 
on the subject. Lectures are given in the very 
first block of 4 weeks of the first year for Civil 
Engineering student*,. At the same time only 
one other course Is taught in that block. 

The learning material consists of a book and 
an additional study guide, indicating the topics 
which are dealt with in the lectures and the 
exercises which the students have to do at 
home. 

Out of the 80 hours which are available for this 
subject, 20 are devoted to lectures for a group of 
approximately 300 students. In the remaining 
60 hours students are supposed to study by 
themselves. To stimulate this self activity five 
computer tests have been Implemented which 
were originally obligatory for all students. 
Each test concludes a chapter and has to be 
done apprax. 2 days after the chapter has been 
dealt with In the lecture. The first test has to be 



taken In the very first week of the first year. 

Four groups of approx. 75 students are formed. 
Each group has one and a half hours to 
accomplish the test and as a result in 2 
afternoons the testing of 1 chapter for a total of 
300 students is complete. A written 
examination concludes the subject. The final 
mark for the subject Is obtained for 70 % by the 
written examination and for 30 % by the tests. 
The written examination must have a 
minimum of 4.5 out of 10, otherwise the test 
result doesn't count. The tests can only be done 
once and the test result Is only valid for the 
subsequent written exam. When the students 
are working on the tests they are allowed to use 
their book and all the other study materials; 
when they do the written examination they are 
only allowed to carry a pen, a pencil and a 
pocket calculator. 

The only difference wJth the courses M and Mil 
is that FM is taught in 2 blocks (8 weeks. 120 
hours) and that both are taught at the end of the 
second yearfnot parallel). 




Question* 



A narrow beaut BC with a length of 
% - 2.5 m and a mass of m = 4 kg 
can rotate freelg around an axis A 
which lies a distance ft/ A from the 
end of the beam B. The beam is 
released from an angle ft - 45*, 
with no initial velocity, 
g = 18 m/s* . 
There is no friction. 



a) Determine the moment of inertia 
of the beam around A. 




Figure 1. Example of a question ir the computer tests. 



ERIC 



89 



105 



The tests. 

Every computer test consists of approx. 7 
questions. Every question requires one 
numerical answer from the student. An 
example of such a question is given in figure 1. 
The student is allowed to answer three times. 
The first wrong answer may be a calculating 
error and will subtract only 1 point from the 
maximum of 10 points which can be obtained 
per question. The second wrong answer "costs" 
the student 3 points. The student can also get 
help if he wants. This help consists of 
instructions which guide the student to find the 
right approach to the solution of the question. 
If the student asks for help he loses 3 points. 
When the student answers wrong for the third 
time he loses all points. After that the 
computer immediately gives the right answer. 

Just like in a "normal" test the student can stop 
a question and return to it afterwards. The 
computer stores the status of the question In its 
memory. 

To avoid that students get the same questions 
each problem type has four alternative 
questions of the same type, out of which 
randomly one is chosen. In addition the 



computer randomly chooses values for two 
variables, between a lower and an upper limit. 
This makes every answer to a question 
practically unique. This answer Is calculated 
by the computer in real time by means of an 
algorithm. 

Results. 

In the MI course the regular computer testing 
has been used for 2 years; in the FM and Mil 
courses for only one year. 

In table 1 we present the data concerning the 
participation of students on the tests and the 
written examination, the results of the final 
examination and the transfer rate after the 
subsequent examination. These data have been 
compared to the situation in which the course 
was taught In the "classical" way, without the 
computer tests. In the academic year '88/'89 the 
MI course was conducted in this way for the last 
time. It was then spread over 2-8 weeks, along 
with several other courses that were running 
simultaneously. In the year '89/'90 the FM and 
Mil courses were conducted in the "classical" 
way for the last time. 





M I 


F M 


M n 




88/89 


89/90 


90/91 


89/90 


90/91 


89/90 


90/91 


Total number of students 


310' 


250' 


300' 


230» 


185 J 


230" 


185' 


Number of participants in tests 




237 


280 




105 




105 


First-time participants 
subsequent examination 


188 


228 


254 


107 


120 


70 


120 


Result of sub- > 5 (pass) 
seque" '*• ■ 5 (comp. 1 ) 
examination < 5 (fail) 


64 ft 
11 * 
25 * 


66 ft 
14 ft 
20 ft 


64ft 
15 ft 
21 ft 


59 ft 
12 ft 
29 ft 


74 % 
18 ft 
8ft 


81 ft 
7 ft 
ft 


76 ft 
9 ft 
15 ft 


Transfer rate (overall) 


42 ft 


73 ft 


65 ft 


30 ft 


69ft 


26 ft 


59 ft 


Transfer rate (pass only 
on written examination) 


39 ft 


60 ft 


54 ft 


27 ft 


48 ft 


25 ft 


49 ft 


Improvement factor 


— -> 1.5 1.4 


---> 1.8 


— > 2.0 



' 1" year; ! 2 - year, estimated; • 5 means: 5 out of 10, can be compensated. 



Table 1. Data for the courses Mechanics I (M 0. Fluid Mechanics (F M) and Mechanics II (M II). 
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If we compare these data, we have to be careful 
to do an "honest" comparison. Taking the test 
result Into account does affect the final mark 
for the course. Therefore we have also included 
the transfer rate of students who passed the 
written examination only. The level of the 
questions in the final examination has not 
been affected by the different way of running 
the course. Hence we assume the comparison to 
be fairly honest. It is striking that the transfer 
rate (pass only on written examination), 
defined as: number of students who passed the 
written exam divided by total number of first 
or second year students, has increased 
significantly for all 3 courses, whereas the pass 
rate for the examination . defined as: number 
of students who passed divided by number of 
students who participated in the examination, 
remained nearly constant. The increase of the 
transfer rate can be mainly ascribed to the 
strongly increased participation of students on 
the written examination. Apparently the chang 
es in the way the course was taught cause a 
decrease In students neglecting their regular 
work for the course. An important remark we 
wish to make i* that it cannot be fully 
determined whether this is only caused by the 
introduction of computer tests or also by the 
introduction of block/modular teaching. If we 
compare the data from the 3 mechanics courses 
with other courses that have changed to the 
system of block teEChing, but which use no 
computer testing, we see that for the latter the 
transfer rates have not increased. 

Effects on study behaviour. 

Of course the time that students spend on the 
course Is an important factor. According to the 
norm the students are supposed to spend 
approx. 80 hours on the MI course. From time 
measurements in '89/'90 it appeared that 
students spent an average of 75 hours on the 
course. This can be called satisfactory, 
especially taken Into account that the number 
of contact hours was limited. For the FM and 
Mil courses the figures are equivalent. Students 
express a very positive attitude towards the 
computer tests. In a written inquiry they 
clearly Indicate that the computer tests 
stimulate them to work regularly on the course. 
Competition between students Is stimulated by 
the fact that students do the tests together In 
groups of 50 and their results are immediately 
available. Inside the test room students work 
conclously and outside the test room vivid 
discussions arise between students about the 
right solutions to the questions. The students 
are working actively on the tests In an 
atmosphere of mutual competition. The 
increase in student motivation, as compared tc 
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previous years, is also notlceble in the lecture 
theatre. Apart from motivation, competition 
also increases stress. To a certain extent stress 
can work positively on motivation, but we ask 
ourselves how far one can go. It does not seem 
desirable to implement this system for all 
courses In the first and second year. 

The tight organization of the course (the tests 
have to be done in groups at a scheduled time) is 
experienced as very positive by the students, 
and not at all as scholastic . On the other hand 
It urges the lecturer to organize the computer 
tests perfectly, because students are distracted 
easily by small weaknesses In the organization 
of computer aided testing, especially because 
they take It very seriously. 

Conclusions. 

The conclusion seems Justified that these 
computer aided tests are a good and effective 
means to stimulate students to work regularly 
on mechanics course. The positive results of 
the system can be ascribed mostly to the 
computert tests, together with the Introduction 
of block teaching. Block teaching Itself, 
without computer tests, does not increase 
transfer rates. Students react positively If the 
system is implemented in several courses In 
the first and second year. The system has a 
positive result on the transfer rates, or, more 
generally, a more regular way of studying can 
be stimulated throughout the whole curriculum 
by these computer tests. It requires a good 
attuning of learning materials, lectures, 
computer tests and written examination. Hence 
a perfect organization is necessary. Finally it 
Is our belief that, for courses like mechanics, 
the computer tests give the student a good 
alternative If the personal assistance from an 
instructor in tutorials is not available. 

Literature, 

Information on the authoring system used to 
produce the computer tests can be found in: 

Laagland, E.F. (1986). TAIGA, a graphic 
authoring system . In: Proceedings of the 27th 
ADCIS Conference on Computer-Based 
Education and Training (pp. 84-89). New Orlea 
ns, LA 
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SETTING THE SCENE: SIMULATIONS IN CBT 



Undo. Blumstein and Rich Boyer 
The Emdicium Group, Inc. 



Abstract 

CBT has proven to be an effective medium for 
training. However, it achieves its full potential 
when simulated scenarios are incorporated 
into lessons. Scenarios provide realistic 
situations and allow learners to practice tasks 
within specific Job functions. This paper 
discusses tbr theory for using scenarios within 
CBT and specific examples from actual 
courseware. The examples include data entry 
functions, a product planning system, a 
customer service situation, a publishing 
system, and a collections system. 

Introduction 

Developments in technology over the past few 
years have produced many new ways of using 
computers to provide training and improve job 
performance. In this paper we first provide a 
brief review of literature defining what 
simulations are and how they are used to 
improve learning of job-related skills. Then 
we show how we've been able to incorporate 
simulations into actual training situations. 

According to Niehoff and Romans (1987), 
training programs have traditionally provided 
transfer of knowledge and skills. When 
applicability on the Job was largely 
unsupported, then knowledg and skills were 
lost. "Research has shown that learning by 
active participation is much more effective 
than listening to lectures. Adults, especially, 
want meaningful, practical learning 
experiences which they can put to work 
immediately" (Niehoff and Romans, 1987, p. 3). 
The content is important to adult learners. 
They want to work on solving specific 
problems and finding real solutions. When 
this happens there is a greater likelihood of 
skill transfer to the Job. 

As microcomputers become more powerful, 
realistic scenarios are being designed and 
developed for an increasing number of 
industries using computer-based training (CBT) 
simulations. Corporations, Insurance 
companies, mar ufacturers and finance 
companies use simulations that "feature 
decision making and management skills in 
realistic scenarios" (Haslam, 1990, p. 10). 
Banks and credit companies are training 
tellers in on-line tasks via simulations. 



What is a simulation? According to Spannaus 
(1978) "simulations operate under fixed rules, 
relationships, and models" (p. 16). Haslam 
(1990) states that "a simulation is an operating 
model of the central features or elements in a 
real or proposed system, process, or 
environment" lp. 10). The model for a 
simulation must contain the relevant cause- 
and-effect relationships, producing 
consequences as a result of the student's 
actions or decisions. In a simulation, learners 
become aware of, and deal with, constraints 
and demands simultaneously. A simulation "is 
dynamic as opposed to static; includes only 
selected elements of the referent system; and 
includes different types of referent systems" 
(Haslam, 1990, p 10). 

Simulation participants interact with the 
setting by making decisions and dealing with 
the consequences of their actions. Simulations 
include "playing roles, achieving goals, 
performing activities with constraints and 
payoffs, and linkages with all of these" 
(Haslam, 1990, p. 10). In order for the activities 
to be of value, learners must be aware of the 
relationships. The simulation activity must be 
related to real life, it must be based on a model 
of reality. The objectives of the simulation 
"must be at the level of application" (Spannaus, 
1978, p. 17). 

The use of scenarios in computer-based 
training simulations allows the learner to 
"devise strategies to solve probkris and tap 
opportunities" ("Management Training," 1987, 
p. 58). Scenarios written for computer-based 
simulations allow us to use computers to 
"improve specific work-related performance 
and productivity" (Ganger, 1989, p. 116). 
Effective skills training usually involves 
training that simulates the job or skills to be 
performed. Formal computer-based training 
using realistic scenarios In simulations relates 
directly to this concept. 

The following examples, created by The 
Emdicium Group, Inc., demonstrate the use of 
scenarios in CBT simulations that is directly 
applicable to Job skills needed by the learner. 
In these examples the learner is actively 
participating in his/her learning and solving 
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Job-related problems, as well as gaining 
practical skills. 
Examples 1 and 2 

By using realistic scenarios when developing 
CBT simulations, we provided the learner with 
specific Job-related problems for them to solve. 
Ganger (1989) describes the use of a computer- 
based training simulation with "maximum 
transferability to actual work procedures." The 
following CBT simulations were developed to 
train data entry procedures. Scenarios were 
created using simulated data so that training 
mistakes did not affect the integrity of the real 
database. But the procedures taught wer* real 
because the training medium was the same as 
the work medium. 

Health Care Benefits 

At a major manufacturer, personal benefits 
representatives (PBRs) were sending paper 
forms to the company's insurance carrier to 
change Information on employees' benefits 
records. The Insurance carrier offered a new 
system that let PBRs access the carrier's 
computer system and enter the changes 
themselves. 

Computer-based training was used to teach the 
PBRs to convert the manufacturer's codes to 
codes used by the insurance carrier using a 
Jobaid provided by the Insurance carrier and 
enter changes to an employee's record using the 
form filled out by the employee. 

We developed a printed Learner's Guide to set up 
scenarios involving common changes that 
PBRs would encounter. The guide contained 
reproductions of forms filled out by employees 
and portions of the Jobaid for conversion of 
codes. 

Use of scenarios that directly related to the 
PBRs' Jobs enabled the PBRs to leam and 
become product re with the new system in 
about three hours. 

Publishing 

A publisher was implementing a major new 
system to handle Its customer and advertising 
databases. Implementation involved 
reorganization of workflow and departments, 
and totally changed the way in which many 
employees performed their Jobs. Furthermore, 
training for the system had to take place during 
negotiation of a new contract with the 
employees affected by the new system. 

To ensure that employees had enough practice 
to feel comfortable with the new system, we 
produced about 4,000 scenarios to be used with 
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computer-based training and a training 
database. Learners attended 30 two-hour 
sessions scheduled across a four-month period. 
At each session, learners received 20 to 40 
minutes of computer-based training and then 
practiced with a packet of scenario exercises on 
the training database. 

During the training, learners managed to 
overcome extreme resistance to the change 
occurring in their workplace. The heavy use of 
scenarios allowed them to fully understand the 
change and prepare for it before the new system 
was implemented. The intensive practice with 
a training database in short sessions over a 
long period of time allowed learners to 
gradually adapt to the new system and work on 
accuracy and speed. 

When the system was implemented, users were 
able to switch Immediately to their new Job 
responsibilities and maintain the same level of 
productivity. Even when problems arose from 
Incorrect or incomplete data conversion from 
the old system to the new, users were able to 
quickly Identify the problems and devise 
solutions using the new system. 

Examples 

According to Clements (1988) relevance of 
scenarios to the target population Is also of 
paramount importance. Clements uses the 
term case study which we refer to as scenarios. 
"In order to provide relevant simulation to 
each employee subpopulatlon, a case study 
bank must be developed concurrently with the 
training. Many cases may be universally 
relevant to major subpopulatlons such as 
hourly or salaried employees. Universal case 
studies are those In which the root 
circumstance is familiar to the overall 
population. Case studies must also be Included 
which represent the unique or high-risk 
circumstances peculiar to smaller 
subpopulatlons" (p. 9). In the following 
example, we created a simulation that allowed 
different audiences to access only those 
scenarios that were relevant to their specific 
Job functions along with a universal scenario 
which was relevant to the overall population. 

Product Planning 

A manufacturer was preparing to implement a 
new system for the planning of new products. 
The system would bring together information 
entered and used by nine different Job 
functions In five departments. Users would be 
scattered In many locations throughout the US, 
and would use the system to communicate all 
decisions and Information concerning the 
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planning of any new product to be offered by 
the company. 

Although each one of the nine job functions 
used only a portion of the new system, users 
needed an understanding of the overall flow of 
the system and the interaction among tasks 
performed by the Individual job functions. 

We developed a universal scenario to first 
present the system flow and Interaction among 
tasks. We created a brief video showing the 
Involvement and decision making for the 
various job functions. We then developed CBT 
to guide learners through the progression of 
tasks performed by all functions. The CBT 
simulated the planning of a plausible new 
product from start to finish. 

After the overview of the system, each learner 
took additional CBT lessons that gave detailed 
Instruction on the specific tasks to be 
performed by the learner's Job function. 
Instruction for the tasks used the same 
universal scenario from the overview, thus 
allowing learners to review the overall flow 
and to learn how their specific tasks affected 
others using the system. 

Example 4 

Smith (1988) discusses the need for assessing 
competence In order to provide accreditation 
for vocational qualification. He states that "to 
carry out such a wide-ranging assessment on- 
line In a high-speed, high -volume continuous 
process Is unlikely to be considered a practical 
proposition, especially as regards assessing 
competencies in fault diagnosis and 
correction" (p. 68). In another article 
("Management Training," 1987) the author 
indicated that "business simulation programs 
allow companies to develop and test 
management skills In a risk-free context" (p. 
58). In the following example we assessed skill 
competency by creating a simulation of a 
realistic situation to evaluate the learner 
performance. 

Collections 

A finance company was Implementing a new 
on-line system to track collection of 
delinquent accounts. Users had to Interact 
with the system while they were talking with 
customers on the phone. 

Because of the sensitive nature of the tasks 
involved, the company wanted to be sure 
learners could correctly interpret information 
received over the phone and enter the 
information, correctly coded, Into the system. 
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In order to make the evaluation as realistic as 
possible, we scripted and recorded a series of 
phone conversations on tape. After listening to 
a scenario on the tape, the learner had to use 
the simulated system to inquire for specific 
information and enter data related to the taped 
phone conversation. Using the CBT 
simulations and taped conversations, we were 
able to evaluate the learners' ability to use the 
new system properly and correct for errors 
without affecting a real customer. 
Example S 

Smith (1988) also points out that learning 
takes place as a result of "doing something, 
thinking about It, drawing out general 
principles from the experience and finding a 
way of testing conclusion" (p. 67), Activity is 
essential for effective learning to take place. In 
the following example we provided activities to 
help learners develop and consolidate sound 
conceptual understandings of their job and the 
required performance by simulating it, and 
allowing "free play" with parameters affecting 
the system. 

Customer Service 

A company was replacing the computer system 
used In its pharmacies. The new system had a 
consistent, intuitive design that allowed users 
to complete 90% of their tasks from one screen. 
Because of the excellent system design, training 
could be kept to a minimum, allowing learners 
to attempt simulations of tasks and receive 
Instruction only when necessary. 

We developed computer-based training 
employing a story line of the pharmacist's 
"first week" of using the new system. By 
simulating a week in the pharmacy, scenarios 
could logically cover common activities as well 
as those occurring only occasionally. 

On the first "day" of the simulated week, 
scenarios Introduced all the common features 
of the system. In these lessons, the Instruction 
was overt, making sure that all learners 
received the basic Information necessary to use 
the system. 

On subsequent "days" of the week, scenarios 
provided learners with the stimulus for tasks 
(prescriptions, health care cards, etc.) and 
allowed them to practice tasks within a 
simulated environment. If learners completed 
the tasks correctly, no prompts or Instruction 
was given. If learners experienced problems or 
requested help by pressing the system help key, 
the simulations would then display Instruction 
specific to the task. 
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Summary 

As the above examples show, the use of 
scenarios in CBT to create realistic 
simulations offers a number of benefits to the 
training situation. The learning process is 
dynamic, not static and therefore the learner is 
involved in his/her own learning as a 
participant. Active participation allows the 
learner to take control of his/her own 
learning. Learners are able to develop higher- 
order thinking skills when participating in 
simulations that focus on problem solving. 
They must not only act, but analyze, recall, 
synthesize and create solutions. Use of CBT 
simulations reduces training time as learners 
become competent in relevant tasks and job 
skills in a shorter period of time. The 
reduction in training time also reduces 
uaining costs over time. Also, built-in 
assessments pinpoint areas that need 
improvement and therefore training becomes 
cost efficient. Use of realistic scenarios in CBT 
simulations improves learner performance 
and decreases training costs and time. 
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QUANTITATIVE AND QUALITATIVE EVALUATION IN PLIGHT TRAINING 



Tom Downey, Boeing Commercial Airplane Group 



ABSTRACT 

This paper deals with a research field trial at 
Boeing Commercial Airplane Group. The 
question we attempted to answer here is 
whether CBT training delivered in a crew (dyad) 
mode is more efficient, effective, and perceived 
more positively by the student than CBT 
training delivered in an individual mode. 
Implications for Computer Managed 
Instruction (CMI) will also be addressed. 

STATEMENT OF THE PROBLEM 

Computer Based Training is utilized by the 
students at Boeing Commercial Airplane Flight 
Training on a one student one terminal basis. 
Their actual work environment is a crew of 
two. This presents a question as to what is the 
best way to train students, in an individual 
environment orina crew (dyad) environment. 

There are several factors that one must take 
into account when making a decision to change 
a training delivery method: 

1. More efficient use of the Computer Based 
Training hardware. 

2. More effective instruction through the use or 
availability of contingent feedback as a 
direct result of pairing students into crews. 

3. Relnforcement of cooperative learning and 
cooperative work skills. Thfc, is in support 
of the AQP from the FAA. The FAA will be 
requiring new approaches to training, one 
of these is training in Cockpit Resource 
Management (CRM). 

4. The cultural factor, approximately 80% of 
the students come from non-English 
speaking countries 

5. The affective component. How will the 

students feel about this type of training. 

6. The effect of the Advanced Qualification 
Proposal(AQP) from the Federal Aviation 
Administration (FAA). 

The mission of this training organization is to 
provide the most efficient and effective 
training possible. This is a great responsibility 
as the recipients of this training are 
responsible for thousands of lives every day. 

The question that this research will attempt to 
resolve is the following: 

Training with CBT In a crew mode (dyad) Is 
more efficient, effective, and perceived more 



positively by the student than CBT training 
delivered in an individual mode. 

LITERATURE REVIEW 

A review of research literature finds a number 
of studies conducted in: 
-Cooperative learning 

-Dyad/triad groups on Computer Based 
Training/Computer Assisted Instruction for 
children 

-Crew training for tank operators in the army 
on CBT or simulators 

-Flight simulator framing for aircrews in the 
military and civilian environment 

Nothing specific was found about the use of CBT 
in a crew mode for pilot training. 

The review was directed to find all applicable 
articles that were directly or tangent tally 
related to the problem statement. Implications 
are listed at the end of each review to relate 
findings in the literature to the problem 
statement under investigation. 

SUMMARY OF FINDINGS 

1. Students working in pairs worked 
significantly more problems on the CBT than 
those working individually on CBT. (Fisher et 
al. 1975) 

Implications: 

Working as a crew could be more efficient and 
take less time to accomplish the same tasks as 
an individual learner. 

2. Pairs that work together and vocalize (talk 
through) the problem solving or concept 
learning strategies necessary for 
accomplishing a learning task appear to have 
higher mastery of the material and better 
retention than individuals or pairs who do not 
vocalize. The ability to explain to or work with 
a partner allows the learner to form his or her 
own strategy for problem solving. This allows 
for longer retention of information and more 
accurate recall of information. Instructional 
time is more efficiently used by individuals but 
more effectively used by pairs when problem 
solving is considered. (Durling et al., 1976), 
(Johnson et al., 1985), (Carrier et al.. 1987) 
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Implication*: 

Crew training seems to be more effective in the 
areas of problem solving and enhances 
retention of training. Efficiency In terms of 
time on task of the crew based system may be 
less but seems to be offset by the effectiveness of 
this approach over the Individual approach. 

3. Student vocalization and interaction during 
CBT with the second person acting as a 
confederate (learning together) or a teacher 
(one teaching the other) was more effective 
than vocalizing to an experimenter (someone 
not involved in the process with the student by 
either supporting the student or being taught by 
the student). (Carrier et al., 1987) 

Implications: 

Talking through the procedure or task during 
the instruction session with your partner is 
more effective than explaining it to someone 
not directly involved with your task. 

4. Contingent feedback (information about 
what you did wrong, why and how it should 
have been done better) was more effective than 
generic feedback (No, try again, etc.). (Gilman, 
1969) This type of feedback could be supplied by 
the lesson or by a confederate who has more 
content knowledge. 

Implications: 

Designing this type of beneficial feedback Into 
a program is costly. Creating an environment 
for this type of feedback through crew training 
could allow this type of feedback to occur 
naturally through interaction between 
members of the crew. 

5. Students working in pairs requested 
contingent feedback more often than those 
working alone. (Carrier et al, 1987) 

Implications: 

Asking for specific feedback may imply that 
you are cognitively processing the information 
at a deeper level. Students working together 
appear to be spending more time processing 
this information, thereby contributing to the 
transfer and retention. 

6. Studies of three/four students per terminal; 
two per terminal; and one per terminal found 
that the three/four per terminal groups 
completed the training ."aster and with less 
variability in their training times than 
students in other conditions. (Okey and Major, 
1976) 
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Implications: 

Grouping students together may allow them to 
finish at a more standardized (homogeneous) 
rate than individually or in smaller groups. 
This has implications in scheduling training 
sessions. 

7. Individuals in groups learn as well as 
individuals working alone with conventional 
computer-based training, however, they take 
somewhat longer to complete each 
lesson/module. In spite of this, large cost 
savings resulted since the group took 
significantly less time than had each 
individual working alone. (Cartwright, Glenn, 
Cohen, Penny, 1977) 

Implications: 

Consensus appears to have a positive effective 
on a crew-based CBT configuration. Training 
in a group may be more efficient. 

8. Individualized learning can take place in a 
group setting through the use of differential 
interpretation of feedback. The student 
maintains his/her own response and compares 
it to the feedback offered by the computer. In 
this way learning may occur among several 
members of the group as they are able to test 
their own hypothesis independent of the group. 
(Cartwright, 1976) 

Implications: 

The lesson should be designed to accommodate 
students who want to test their own hypothesis 
while working in pairs or In groups. This may 
prove to be an inefficient method of providing 
feedback to the learner. 

9. Peer tutoring and cooperative learning have 
yielded significant gains in increasing 
academic achievement. Cooperative learning 
benefits both the high and low achieving 
students. (Moersch, 1987) 

Implications: 

The main principle in cooperative learning lies 
in the philosophy behind positive 
Interdependence. This goal emphasizes the 
need to be concerned about the performance of 
all the group's members as well as the 
individual's. Individual accountability 
ensures every student's mastery of assigned 
material is assessed, each student is given 
feedback on bis or her progress and the group is 
given feedback on how each member is 
progressing so that the other group members 
will know who to help. 

This approach has positive implications to the 
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Issue of AQP and CRM training. 

10. CBT delivered In a crew mode was more 
cost-effective than Individualized CBT. 
Training time was significantly lower and less 
variable for the crew-based CBT students than 
for the students In the individualized mode. 
There seems to be some disagreement regarding 
students of low ability. Schlechter in his 
report cited Morrison, (1986) who found that 
individualized training elicited better 
performance with low ability students than a 
crew-based approach. He also cites Flagman 
and Rose (1983) who state that the low ability 
students tend to benefit more from crew-based 
than individualized. (Schlechter, 1986) A well 
designed Computer Managed Instruction (CMI) 
package that collects both quantitative and 
qualitative data would help to resolve this 
Issue. 

Implications: 

Pre-tests may be given to students with the goal 
of separating out the low ability from the high 
and palling them equally. This could allow the 
lower ability student to have the type of 
differential feedback that would be necessary 
to learn the material and give the higher 
ability student the opportunity to process the 
information more deeply by explaining it to 
the lower ability partner. There is a 
frustration factor that will have to be dealt 
with here, some higher ability students will 
feel held back If matched with a lower ability 
student and may object to this. This could be a 
subject for further research. 

METHOD 

The decision to use a Crew Computer Based 
Training approach should be supported by 
empirical data collected by an assessment of 
the performance and attitude of the students In 
a crew and individual setting. The attitudinal 
assessment will be done with the same group of 
students In both individual and crew 
approaches. 

Proposed assessment procedure: 

a. Select lessons to be used in assessment. In 
this case all the CBT lessons (71) will be 
used. 

b. Design assessment approach and tools. 
There were three tools designed for this 
study: 

1. A demographics form 

2, An attitudinal survey form for the crew 
and individual approaches. 

3 Ji Computer Managed Instruction (CMI) 
Program 

4.A qualitative data gathering capability 
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will exist with the CMI program to 
allow the students to leave individual 
or crew comments on the lesson or 
particular frame (screen of 
information). This will allow more 
accurate analysis of their areas of 
concern. 

c. A field test was done with Subject Matter 
Experts and Leads to obtain formative 
evaluation feedback on the process and 
instruments. 

d. Identify students to be used in test. Assure 
that they are a representative sample of 
the student population. Students were 
selected from the next five classes to arrive 
at the training center. A total of 40 
students were to participate in this study. 

e. Data gathering was planned to be done by 
observation, survey questionnaire, 
interview and Computer Managed 
Instruction. The CMI program was not 
available as scheduled but the 
Implications of it's use will be discussed. 
The crews were divided into two groups, A 
and B. The first group took the first half of 
the CBT lessons in a crew configuration 
and the second half in an individualized 
configuration. The group B students did 
the opposite. This gave us the opportunity 
to have all CBT lessons taken in both 
configurations. 

f . Analyze data 

Data from the demographics form were 
consolidated for a descriptive treatment of 
the students as a whole. 

Data on performance in the CBT lessons 
was to be gathered unobtrusively through 
the use of the CMI program. This is the 
data that would allow us to examine the 
efficiency and effectiveness of the two 
delivery modes. 

The data to be captured is: 
-Student Identification 
-Time through the lesson 
-Response latency on practice items 
-Number of correct answers in a lesson 
-Number of times the student has gone 

through the Instruction and the practice 

section of the lesson 
-Individual/Crew Comments on the lesson 

g. Make recommendations 
Recommendations will be made after a 
significant number of students have gone 



through the training. 
PARTIAL RESULTS 

The following Is a description of an on-going 
study. Computer Managed Instruction Results: 
The gathering of Computer Managed 
Instructional data has been delayed due to 
limited programmer resources. 

Affective Results: 
N=21 

1. Learning on CBT as a crew was more effective 
(better). 

Range 1-5, X = 3.7,SD= .98 

2. Learning on CBT as a crew was more efficient 
(faster). 

Range 1-5, X= 3.95 , SD = 1.13 

3. Pro-Crew: Open ended responses: (not all 
students responded) 

-Very happy about crew approach 
-Consultation was good 
-Discussion was good 

-Two persons are more active, It's more 
difficult to fall asleep 

4. Learning on CBT as an individual was more 
effective better). 

Range 1-4, X = 2.95 # SD= 1.21 

5. Learning on CBT as an individual was more 
efficient (faster). 

Range 3-4, X - 3.43, SD = .73 

6. Open ended responses: (not all students 
responded) Pro-Individual: 

-I can go at my own speed 

-I can repeat questions 

-I didn't like the crew because my partner 

wanted to go faster. 

7. Overall I liked best (not all responded) 

-Crew 9 

-Individual 7 

IMPLICATIONS AND RECOMMENDATIONS 
EARLY IMPLICATIONS: 

l.The research literature Implies that paired 
(crew)learnlng can be more effective than 
individualized learning when the pair can 
verbalize as part of the learning activity. 

A possible restructure of lesson flow to 
Incorporate procedure activities that require 
verbalization from both crew members could 
enhance retention of the procedural 
knowledge and make for more effective 
instruction. 
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2. Paired learning is somewhat less time 
efficient than individualized learning, 
however there is a larger benefit to be realized 
In the retention of knowledge over time. If we 
are able to double the efficient use of each 
CBT workstation by using the crew concept 
we will more effectively use the resources In 
our CBT instructional area. 

Any incremental increase in training time in 
the individual vs. crew approach could be 
offset by the increase in effectiveness and 
overall quality of instruction. 

3. Crew training would have to be coordinated 
with individualized testing. This could be 
done with a content and procedural 
knowledge computer based testing approach. 

Implications for Using Computer Managed 
Instruction In the Collection of Data 

In using the Instructional Systems Design 
model one of the areas that is usually omitted 
or not done well is the evaluation step. A 
formative evaluation is usually done in order 
to initiate full-up production in a timely 
manner. The summatlve evaluation usually 
left for the end is forgotten or done in a rather 
cursory manner. 

We will now discuss the design and 
development of a Computer Managed 
Instruction system that will allow the 
formative and summatlve evaluation to be 
performed In a timely and objective manner. 

The Federal Aviation Administration (FAA) 
has released an Advisory Circular (AC) that has 
been signed and Is In effect. This AC proposes 
an Advanced Qualifications Program (AQP) 
that will change significantly how the air 
Industry performs the training function. It 
specifies a systematic approach to design and 
development as well as a methodology for the 
validation of training performance through 
the use of performance objectives. 

The tracking of the student performance on 
these objectives must be reported. The 
development of these objectives and the results 
of the student's performance must be trackable 
through the use of an Instructional Audit Trail. 

This section of the paper will describe the 
development of CMI in light of gathering data 
on the crew vs. Individual debate. 

AQP Evaluation requirements 

Evaluation In the terms of the AQP must have 




the following components: 
-Careful appraisal of an individual 
-Standards required for a specified level of 

proficiency 

-Instructional Audit Trail 

•Criticallty Analysis, the impact of 

substandard performance on safety. 
•Performance objectives must be 

systematically maintained, developed and 

validated. 

•Formative and summative evaluation must 
be conducted 

-Use of the Instructional Systems 
Development model 

CMI Definitions 

In order to clarify the function of the CMI 
program and its' component parts some 
definitions are in order: 

Registration: A function that allows a registrar 
to identify students to the system and gtve them 
an identifier to permit access to the system. 
This will also gather some demographic data 
and allow registration of the student to a 
particular airplane configuration. 

Logon: Allow the student access to the CBT 
lesson menu which fits his airplane 
configuration. This also identifies the student 
to the system and initiates the process of data 
collection. 

Course management: This allows the tracking 
of the progress of classes of students by an 
Instructor/administrator and gives individual 
feedback to the student as to their progress 
through the course. 

Student performance: The student's time 
through the lesson, number of correct vs. 
Incorrect responses and their response latency 
will be tracked. 

Data Collection, quantitative, qualitative: 
Quantitative data will be collected on student 
performance (above), qualitative data will be 
collected at the screen level and the lesson 
level. This data will consist of the student's 
comments on the quality of audio, video, 
graphics, technical content level of the lesson 
and any open ended responses the student may 
have. 

Formative evaluation: This is conducted with 
prototype lessons and will give immediate, 
objective data on the way prototype lessons are 
used. 

Summative evaluation: This will give long 
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term or longitudinal data on the performance 
of students and lessons over time and over 
diverse groups of students. 

Reports for management, instructor, 
instructional designer: These will consist of 
student performance, lesson performance, 
course performance and qualitative feedback 
from students for overall courseware 
improvement. 

Clfl Implementation for Flight Training 

Our program has certain specific needs that can 
be generalized to other programs. The ISD 
process allows us to create courseware of a 
consistent qualify. The main need Is to assure 
that this quality is high. The approach to 
improving quality is through meeting the 
following needs: 

-Specific, objective quantitative data 

collection; student performance data, 

courseware management 
-The ability to make informed/accurate 

decisions for future directions of Flight 

Training 

-Allow for adherence to AQP requirements 
(future) 

-Collection of comments, qualitative data 

How CMI meet* Evaluation requirements 

The use of CMI will help us to collect useful data 
and allow us to meet the requirements outlined 
above as well as meeting our specific needs. 
CMI accomplishes this through the following 
functions: 
-Individual performance tracking 
-Ability to set a mastery level for each 
objective or lesson 

-Records / analyzes performance on 
proficiency objectives across all students 
-Instructional Audit Trail 
-Allows for maintenance and validation of 
performance objectives 
-Provides methodology for collecting 
objective data for formative and summative 
evaluation 

Future use of CMI for quality control of 
training 

CMI will be an integral part of our training 
system. There are a number of future areas that 
will be examined in the Phase II and Phase III 
development. Some of these Issues are: 

-Computer-based testing - procedure 

oriented 

-Refinement of revision process for CBT 
-Objective, non-Intrusive data collection for 
research into transfer of training issues. 




Future off can 

CMI can also be used across other areas other 
than strictly in the CBT domain. The future 
training function cannot stop at the classroom 
door but must be carried on (and tracked) In 
other environments and over a longer period of 
time. Some of these areas are the following: 

-Procedures training (FBS/CBT) 

-Maneuver Techniques (FFS) 

-Operations (airplane) 

There are already some approaches in place to 
use flight data recorder data to recreate 
scenarios and replay them in a 
CBT/Simulation session. Further 
investigation in the use of this type of feedback 
to the pilot would be useful. 

The use of an expert system as a monitor /tutor 
in a CMI role would allow more efficient 
presentation of instruction, feedback and 
testing. 
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COURSEWARE DESIGN IN THE TOOLKIT AGE 



Jesse M. Heines, EdD., University of Massachusetts 



Abstract 

The extensive capabilities of today's systems 
have made the development of state-of-the-art 
courseware a formidable task. To make these 
sophisticated capabilities accessible to people 
without extensive programming backgrounds 
and to significantly speed up program 
development even for people with such 
backgrounds rich software toolkits have been 
developed. The Macintosh toolkit has been 
around for several years, but Windows 3.0 for 
DOS systems has only recently come onto the 
market. In the workstation arena, we are even 
seeing high level toolkits become available 
that are built on lower level toolkits, as Motif 
is built on Xwlndows. 

The paper discusses how five standard toolkit 
design elements can be applied to computer- 
assisted instruction applications: pulldown 
menus, popup windows, scrollable windows, 
dialog boxes, and help tools. The author 
strongly encourages the use of these elements to 
standardize user interfaces, whether through 
programming languages or menu -driven 
authoring systems. He feels that doing so will 
allow courseware to obtain the standard "look 
and feel" of other applications and thus help 
students to concentrate on the subject matter 
they are trying to learn rather than the 
mechanics of running the course. 

Keywords: 

computer-assisted Instruction 
user interfaces 
screen design 

courseware design and development 
software toolkits 

Issues In Courseware Design 

In 1983 I authored a small book entitled Screen 
Design Strategies for Computer-Assisted 
Instruction. One of the major issues that book 
tried to address was to provide a basis for 
response to James Martin's 1973 lament: 

As yet, no acknowledged sense of style has 
developed for CAI. ... In the meantime, 
however, some singularly unstylish CAI 
programs are being written (Martin, 1973). 

Style remains one of the major issues in 



courseware design to the present day. While 
human factors research has helped us gain 
insight into the best ways to use design 
components such as functional areas, icons, 
and menus, the capabilities of today's systems 
have expanded so quickly that researchers 
have been hard-pressed to keep up with the 
myriad of developments. The net result is that 
even as our knowledge grows about what 
constitutes good courseware design today, we 
are continually asked to reevaluate our 
knowledge In new contexts. There are far more 
choices for courseware designs today than 
there were when I wrote my book in 1983, and 
the problem is orders of magnitude more 
complex than when Martin found it so 
lamentable in 1973. 

Only one factor remains constant: our desire to 
develop courseware that meets the needs of our 
students. With today's systems it is easy to 
become so enamored of the hardware and 
software capabilities that we lose sight of this 
desire. The "KISS" (Keep It Short and Simple) 
principle is often violated as developers fall 
victim to afflictions such as "font-itls and 
color-itls" (van Dam, 1990) using too many 
fonts or colors on the screen at one time. As far 
back as 1971, researchers found that the 
excitement experienced by instructors in 
preparing programs for classroom use did not 
necessarily transfer to their students 
(Desmaisons etal., 1971). 

The main issue, therefore, is really the same as 
it was when Pressey introduced his first 
"simple apparatus which gives tests and 
teaches" in 1926: how can we harness the truly 
fantastic capabilities of today's systems to 
serve the needs of our students? Clearly, there 
is no simple answer to this question. I attempt 
In this paper, however, to point the way to the 
multiple answers provided by the de facto 
standards evolving out of the programs now 
available for today's systems. 

The Toolkit Revolution 

It has long been argued that to educators 
Interested in using computers in Instruction, 
programming is at best an undesirable chore 
and at worst an insurmountable obstacle. 
Some feel that all instructional developers 
should have at least a basic knowledge of 
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programming, while others feel that such 
knowledge may actually be counterproductive. 

One of the strongest advocates of programming 
is Paul Tenczar, President of Computer 
Teaching Corporation, which markets the 
TenCORE family of products. The TenCORE 
language is a modernized and expanded version 
of Tutor, the language that Tenczar developed 
for the PLATO system in the early 1970s. 
Tenczar argues: 

While authoring systems requiring little or no 
computer literacy can open the field to a wider 
pool of authors, a "programmerless" authoring 
environment is as limited as a doctorless 
hospital. ... Programming is one of the 
elements required by a professional 
courseware team... (Tenczar, 1990) 

Robert Becker expresses a similar view: 
Perhaps the most underrated function on an 
interactive training development team is 
programming. Programming is generally 
equated with implementation an uncreatlve, 
mechanistic class of behavior that is supposed 
to follow Instructional design and development 
in lock step... 

[Some advertisements foster] the idea that 
interactive multimedia training can be created 
without programmers. Because their work is 
merely procedural rather than creative, why 
not automate it by using an authoring system? 
The simple answer Is that programming is not 
merely procedural. It Is also among the most 
creative tasks in multimedia development. A 
project team that does not solicit design 
recommendations and guidance from a capable 
programmer is almost certainly doomed to 
produce mediocrity even with the best 
authoring system in the world. (Becker. 1991) 

Not surprisingly, advocates of iconic and 
object-oriented systems take a somewhat 
different view. One of me strongest advocates of 
programmerless systems is Michael Allen, 
Chairman of Authorware, Inc., which markets 
the Authorware Professional family of 
products. Authorware is an outgrowth of the 
PLATO Courseware Design, Development, and 
Delivery (PCD3) authoring system, developed 
by Control Data Corporation In the mid-1980s. 
Allen claims that: 

Through evolution of an approach that is 
graphic, not based on programming, and uses 
object-based technology, we feel it is actually a 
conservative statement to say the Authorware 
products are at least a generation ahead of even 



the most costly systems, including PLATO. 
(Allen. 1988) 

It must be recognized that although their 
approaches differ, both Tenczar and Allen have 
the same goal: to simplify the process of 
creating quality computer-assisted instruction 
(CAI). Allen is no more an advocate of having 
computer neophytes develop CAI than Tenczar 
is of having courseware developers work In 
assembly language (Allen, 1990; Tenczar, 
1990). On closer examination of their 
respective products, one notices an interesting 
regression to the mean: Computer Teaching 
Corporation now markets TenCORE Producer, 
a tool that provides many of the basic 
capabilities of the TenCORE Language 
Authoring System through a menu -driven 
interface. And among Authorware 
Professional's standard set of icons, one 
notices that within the framework of the 
deceptively simple "calc" icon, one can actually 
write code that looks very much like C. In 
addition, both systems have long 
acknowledged the desirability of being able to 
call out to and be called by external programs. 

These developments are extremely important 
to courseware developers. We certainly want to 
use the powerful capabilities of systems such as 
TenCORE and Authorware Professional for 
organizing instruction, managing multimedia 
presentations, evaluating student input, and 
tracking student progress, but we cannot afford 
to be "locked out" when we need additional 
capabilities that these systems do not yet 
provide. We need to be able to pick and choose 
among all options for Improving the quality of 
our on-line instruction. As Gloria Gery (1986) 
has said, M I want it all!" 

This is where the concept of toolkits comes in. 
Software development on today's systems is 
indeed a highly skilled task. It is so 
complicated, in fact, that only the most 
diehard bits and bytes programmer would 
relish writing a program with a state-of-the-art 
user interface "from scratch." Imagine writing 
a Macintosh mouse device driver and a 
complete set of menu subroutines each time you 
wanted to write an application that uses a 
pulldown menu! To avoid this, the Macintosh 
provides a "toolkit" of such routines that high- 
level programs can call. A similar toolkit is 
now provided for DOS systems in the 
Microsoft's Software Development Kit for 
Windows 3.0. In addition, we are now seeing 
higher-level toolkits that make these routines 
even easier to use. For example, it wasn't long 
after MIT made the Xwlndows toolkit available 
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to workstation programmers that a higher 
level toolkit was developed: Motif. 

At the user level, the toolkit concept is also 
critically important, because few people today 
use only one software package. As the number 
of packages that any one person uses on a 
regular basis continues to grow, it becomes 
increasingly important that these packages: 

(1) share a common user interface so that it is 
easy to move among them, and 

(2) can exchange textual and graphic data so 
that redundant efforts can be eliminated. 

Macintosh users have long enjoyed relatively 
seamless boundaries between software 
packages, and DOS users are now starting to 
enjoy the same benefit with Microsoft Windows 
3.0. Donald Norman has stated: 

... the development of the internal toolbox has 
caused developers to be consistent to their use 
of screen, mouse, and key-board, even when 
their natural tendencies would have led them 
elsewhere. As a result, most new programs can 
be used immediately, with little or no study, 
often without even opening the manual. 
(Norman. 1990) 

Thus, programming toolkits have made 
possible application toolkits, and already 
some users are demanding that all applications 
they consider for use conform to high-level 
toolkit standards for user interfaces and data 
exchange. When courseware applications 
achieve these goals, they can become part of the 
complete set of software tools that people in all 
levels of education and training need to get 
their Jobs dene. On-line instruction can now 
become part of a total application package 
rather than a separate add-on. 

Toolkit Design Elements 

Is it possible to serve the dual masters of 
student needs and toolkit standards? I think it 
is. Today's toolkits are rich in interaction 
styles, most of which lend themselves very well 
to instructional purposes. The window 
metaphor is particularly useful, as 
Instructional designs often call for overlaying 
graphics or the output of other programs with 
exp'-natory te.rt. Let us look at five basic 
toclKit design elements to examine how they 
can be applied to instructional sequences. 

Pulldown Menus 

Virtually all instructional programs provide 
their users with choices via menus. Some 
common applications of menus are: 



•selecting the unit one wants to study within 

a specific course 
•switching to another course 
•exiting the courseware system altogether 
•requesting help on using the courseware 

system 

•requesting help on the course subject matter 
•looking up words in a glossary 
•responding to multiple choice test items 

Over the years, courf. ware developers have 
come up with a large number of creative ways to 
present options such as these to students. 
Indeed, many different techniques have been 
shown to be effective. With the advent of 
toolkits, however, the use of pulldown menus 
has far outstripped all other techniques. In this 
technique, a list of main menu options (called a 
"menu bar") appears horizontally across the 
top of the screen (or window). When an option is 
selected, a list of suboptions appears below the 
miln option (see Figure 1). 

This type of menu is probably so familiar to 
most readers that they wonder why I am even 
mentioning it. That Is precisely the point. This 
type of menu is so familiar because so many 
programs use it. It is a de facto standard, and it 
can be generated by routines available in 
virtually all menu toolkits available on DOS 
and Macintosh systems. While It is impossible 
to argue that this is the only viable format for 
designing menus today. I can argue that unless 
one has a specific reason that warrants 
designing menus differently, one should use 
this format. The advantage is two-fold: 

(1) the menu format is so familiar to most 
students that they need no training to its 
use on the contrary, many students now 
consider use of menus in this format 
intuitive, and 

(2) the format is easily implemented through 
routines available in most toolkits. 

Not only is the menu format shown in Figure 1 
a de facto standard, but the words chosen for 
the options, the order In which they appear, 
and their associated "hot keys" (underlined in 
the figure) have also become standardized. The 
interested reader may find extensive 
discussions of these issues in, for example, 
Apple's Human Interface Guidelines (1987), 
IBM's Common User Acess Advanced Interface 
Design Guide (1989), and th? Open Software 
Foundation's Motif Style Guide (1990). Note 
that these guidelines also include standards for 
"pullright" menus submenus of the Individual 
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entries on the pulldown menus. 
It is interesting to consider what options 
courseware designers should put on their 
menus and what actions these options should 
take to adhere to these toolkit standards. 
Following is one possibility. Each of the 
boldface words listed below would appear as an 
option on the menu bar, with the words in 
normal text appearing as options on their 
pulldown menus. 



Edit Search Help 




Save 
Save As... 
Erint 

Page Selup... 
Prjnter Setup... 



Exit 



Figure 1. A menu bar and pulldown menu. 
File 

New switch to a different unit or 
course 

Open temporarily switch to a 
different unit or course with 
the possibility of returning 
to the current unit and 
course 

Save save all data stored on my 
work up to this point so that 
I can restart this course at 
this point after I exit 

Save As let me specify the name of 
the file In which my student 
data will be stored 

Print provide a hard copy of the 
current page or other data 
relevant to the course 

Page adjust various screen Setup 
parameters such as 
colors and type sizes 

Printer let me specify the type of 

Setup printer I want to use 

Exit leave the courseware system 
entirely, with the system 
saving the data on my work 
if it has not already done so 

Edit 



Undo ignore my last response and 

let me try again 
Copy copy the material I have 

highlighted to a file so that I 
• can examine it later, perhaps 

with another software tool 
Paste take input from a file I have 

already created 
Calculate give me a calculator that 

I can use 

Call let me call another program 
and return here 

Support 

Detail provide additional details on 
this topic 

Example show an example of the 
concept being discussed 

Summary summarize this section 

Exercise p/esent exercises related to 
the current section 

Test test me on this material 
Help 

Subject I don't understand this; 

explain the current subject 
matter in another way 

Index show an index of all help 
topics 

Keyboard provide help on using the 

keyboard and the meanings 

of any special key stroke 

combinations 
Commands provide help on the 

commands available to me 

at this point 
Using Help provide help on using the 

help system 
About tell me the date and version 

number of this course 

All of the items in this menu structure would 
not, of course, be relevant to all courses, and 
others would be needed for some courses. This 
sample structure does demonstrate, however, 
that the major properties of the standardized 
pulldown menu structures we see In other 
applications can easily be adapted for 
computer-assisted instruction applications. 

Popup Windows 

One of the most difficult aspects of courseware 
design Is the management of screen space, 
simply because there is never enough of It. In 
my 1984 book I devoted an entire chapter to the 
Issue of functional areas, a term I used to refer 
to the practice of always putting messages of a 
certain class In the same area of the screen. For 
example, I recommended setting aside 
functional areas for instructions, student entry 
prompts, help feedback, and error messages. It 
wasn't long after the book was published, 



however, that I modified my recommendation 
to include the use of windows (Heines, 1985). 




Figure 2. A popup window showing the 
objectives for the current module. 
Copyright 1985, KJ Software, Inc. 
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Figure 3. A Windows 3.0 popup window. 

Figure 2 shows the use of a window in a course I 
developed on writing. This window is used to 
explain the objectives for the current module, 
overlaid onto a screen explaining how to use 
the arrow keys in a word processor. Note how 
the use of a window in this situation maintains 
the instructional context and lets the student 
relate the Information provided in the window 
to the task at hand. In addition, it would be 
Impossible to reserve a functional area large 
enough to display the information in this 
window. When tbc student presses Enter, the 
underlying screen is restored exactly as it was 
when the student pressed the hot key calling up 
the objectives display. 

The windowing capabilities of today's toolkits 
are considerably more sophisticated. 



Consider, for example, the properties of the 
standard Windows 3.0 information popup 
window show in Figure 3, which displays an 
error message for an improperly configured 
printer. On first examination, this window 
appears merely to contain four visual 
elements missing from the window shown 
in Figure 2:(1) a title bar 

(2) a system pulldown menu at the extreme left 
of the menu bar 

(3) an icon (the exclamation point) indicating 
the type of its message 

(4) an OK button to remove the window from the 
screen 

It has two other Important properties, however, 
that cannot be seen visually but that are even 
more Important: 

(5) it can be moved about the screen through 
standard Windows 3.0 protocols 

(6) it can remain on the screen while the user 
takes the actions suggested it is only 
removed from the screen when the user 
clicks on OK 

This last property is critically important in 
instructional situations, where If students 
request help, they typically want it to remain 
on the screen while they answer a question. 

Scrollable Windows 

Another common courseware design problem 
raised by my 1984 book is that no matter how 
interactive and graphic-based we strive to 
make our courseware, at some point we are 
always faced with the need to present more text 
than can comfortably fit on a single screen. In 
1984 I argued against pa < sing between 
successive displays unless the student had a 
clear and easy way to get back to the text that 
was erased when new text was displayed. As 
much as all proponents of interactivity deplore 
excessive press-return-to-continue scenarios, I 
felt that this .technique was preferable to 
erasing information before students may have 
had a chance to digest it. 

Once again modern toolkits come to the rescue 
by providing us with scrollable windows (see 
Figure 4). These text presentation tools allow 
students to view any amount of text in a 
convenient package. They can advance or 
backup through the text a line at a time (by 
clicking on the up and down arrow keys on the 
vertical scroll bar at the right of the window) or 
a screenful at a time (by clicking on the scroll 
bar above or below the position marker). 
Students can also quickly advance to any point 
in the text by dragging the position marker up 
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Figure 4. A Windows 3.0 scrollable text 
window. 



Figure 5. Use of a marker similar to that in a 

scrollable window to provide orientation 
Information on a student's position In a course. 

The marker Is the diamond at the left of the 

line at the bottom of the screen. 
Copyright 1988, Cooper & Associates, Inc. 
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Figure 6. A Windows 3.0 dialog box. 

It Is Interesting to note that the use of a scroll 
bar also provides Important orientation 
information recommended in my book 



students can easily tell how far they are 
through the text by the relative position of the 
marker. This type of orientation Information 
has been implemented In the Teletutor series of 
telecommunications courses from Cooper & 
Associates In Portsmouth, New Hampshire. 
Their Introduction to Telecommunications 
course (1988) used a diamond-shaped position 
marker on a line at the bottom of the screen to 
indicate the student's position In the course (see 
Figure 5). Unfortunately, this course was 
developed before toolkits were available that 
allow students to move the marker and actively 
adjust their position in the course. 
Dialog Boxes 

Today's toolkits handle the majority of 
structured data entry through tools known as 
"dialog boxes." These tools provide a large 
number of interaction styles, three of which 
are shown in Figure 6. 

(1) The Find What field Is a text entry field or 
edit box. The cursor appears as a vertical 
bar and may be positioned using the mouse 
or the arrow keys. 

(2) The Match Upper/Lowercase option Is a 
toggle or check box. Clicking in the box 
turns this option on, signified by the 
presence of an "X" in the box. Toggle box 
options are usually independent; each 
toggle box may be turned on or off without 
affecting the stale of other toggle boxes. 

(3) The Search Direction (Forward and 
Backward) options are radio buttons. 
Clicking in either of the circles turns the 
corresponding option on, simultaneously 
turning all other radio buttons in the same 
group off. Radio button options are usually 
mutually exclusive; only one of each set 
may be on at any given moment. 

Other basic dialog box interaction styles 
Include slider scales, in which values are 
specified by sliding an icon left and right (or up 
and down) on a numeric scale, and menu lists, 
in which choices are presented in a menu and 
the user highlights the option he or she desires. 
Detailed Information on the use of these styles 
Is provided In Murray and Pappas (1990), 
Young (1990), and Open Software Foundation 
(1990). 

The interactions in dialog boxes are extremely 
intuitive, especially when one is using a mouse 
or other such pointing device. In addition, it Is 
easy to find applications of these techniques in 
instructional settings. For example, multiple 
choice questions lend themselves perfectly to 
radio buttons, while questions such as 
"Indicate all possible values of the 
discriminant when A=l, B=2, and C=3" lend 
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themselves perfectly to a set of toggle boxes. Use 
of these toolkit techniques can foster valuable 
consistency between courses, making it easier 
for students to concentrate on subject matter 
rather than the mechanics of specifying their 
responses. 

Help Tools 

The last courseware design toolkit technique I 
want to discuss is on-line help. The help 
available on the Macintosh and Windows 3.0 
today are highly sophisticated software tools. 
In fact, writing help for sophisticated 
applications using these systems is itself a 
skilled art. These help systems allow users to 
get context-sensitive help on the task at hand, 
access an index of all help topics available, 
backtrack to topics they have seen in the past, 
browse forward and backward through help on 
other subjects, and search for help on topics 
containing specified key words (see Figure 7). In 
addition, the text may contain "hot spots" 
graphical or textual on which users may click 
to Jump to linked help messages in a hypertext 
manner. (Such hot spots are underlined in 
Figure7.) 

Help tools will surely continue to develop until 
they evolve into complete computer-assisted 
instruction authoring systems. Indeed, a 
number of the companies with which I consult 
are looking at having their software call 
courseware written in TenCORE, Authorware 
Professional, or other such tools to provide 
learning environments for their products. I 
believe that this level of integration will 
represent a major shot-in-the-arm for 
computer-assisted instruction, and truly allow 
software systems to make possible what Allen 
has called "Just In Time Learning" the ability 
"to provide instruction on the user's selected 
topic on demand" (Allen, 1S89). 

Occasionally one hears the assertion that 
toolkits suppress creativity. "Why should we 
use toolkits that aimost force us to conform to 
a user interface standard that is still 
evolving?" the resisters ask. "Sure, it makes 
implementation easier," they admit, "but the 
price of restriction on our creativity is too 
high." There are two answers to such 
arguments, and both are expressed in the main 
issue I raised at the 

beginning of this paper: "how can we harness 
the truly fantastic capabilities of today's 
systems to serve the needs of our students?" 
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Figure 7. A Windows 3.0 help window. Using 
Toolkits Creatively 
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Figure 8. Use of pulldown menus in a 

computer game. 
Copyright 1989, Sierra On-Line, Inc. 

The first answer is that today's toolkits are 
quickly growing in depth and scope, providing 
an increasingly rich set of user interface 
techniques that can be applied in highly 
creative ways. One of the best examples of such 
creativity may be found in computer games. 
Figure 8 shows a screen from Hero's Quest I 
(Cole et al., 1989), a f Tie which makes heavy 
use of pulldown menus Ualog boxes, and other 
such toolkit techniques uiat game enthusiasts 
move through with amazing speed to get to the 
competition. Such users are seldom interested 
in implementation issues; they simply want to 
know the rules and begin competing as quickly 
as possible. The creativity in these user 
interfaces can be found in their layout, choice 
of type fonts, use of graphics, and the options 
that they provide. The Interaction techniques 
themselves are very secondary especially to 
the 3-12 year old boys who form the vast 
majority of these programs' users but they are 
essentially the same as those discussed in this 
paper. 
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The second answer is that we must constantly 
remind ourselves that the purpose of 
courseware Is to meet the needs of students, not 
instructors. Most students care very little about 
whether they select an answer by pressing a key 
or by clicking a mouse button. They are vastly 
more concerned with the correctness of their 
responses than the techniques they use to 
indicate those responses. The consistency 
provided by toolkits can virtually eliminate 
the need for modules that teach how to take a 
computer-assisted Instruction course, allowing 
students to concentrate exclusively on the 
subject matter. 

We must also keep in mind that the 
programming/ programmerless authoring 
system argument is of absolutely no interest to 
students. Their only concern is with the 
courseware they must try to learn from, 
regardless of how it was developed. It is 
interesting to note that few of us who make a 
living developing courseware have actually 
ever taken a CAI course as a student. We have 
all tried courses out as students, but how many 
of us have actually taken a course for academic 
credit in which CAI is the main instructional 
medium? Courseware development is the 
construction of a piece of software. The 
building blocks are computer subroutines, and 
one may join them together by writing code or 
by arranging icons representing them in a logic 
diagram. The important point is not to 
"reinvent the wheel," but to use standard 
toolkits so that the product has a standard 
"look and feel" and can be completed in time to 
allow revisions when it is tested with students. 

It has been shown that few people lead manua's 
before they use familiar devices such as radios, 
lawn mowers, and microwave ovens. (When 
was the last time you rented a car in which you 
could even found the owner's manual?) The 
user interfaces to everyday devices have 
become so standardized that their use is 
intuitive to most children as well as adults. 
Courseware design based on toolkits will allow 
us to achieve the same intultlveness in on-line 
Instruction. We can then focus on our students' 
needs and spend our time and energy 
experimenting with creative solutions to 
problems dealing directly with their mastery of 
the subject matter. 
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COCKPIT RESOURCE MANAGEMENT TRAINING: MULTIMEDIA SITUATED 

LEARNING 
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Abstract 

Line -Oriented Flight Training (LOFT) for 
flight crews, developed to promote effective 
cockpit resource management, is an example 
of situated learning in a sophisticated 
multimedia training environment. All six of 
the critical situated learning components — 
Apprenticeship; Collaboration; Reflection; 
Coaching; Multiple practice; Articulation — 
are present in the LOFT training program. The 
LOFT program has several important 
implications for designers seeking to 
implement training to accordance with the 
Situated Learning Model: 1) the debriefing or 
reflection process for a team of learners, 
rather than an individual learner must be 
designed to provide feedback for both the team 
and the individual; 2) it is essential to 
differentiate between training and testing to 
the situated learning environment and to 
make sure that this distinction is clear to both 
the instructor and the learner; 3) the role of 
the instructor in a situated learning 
environment must be carefully designed as 
part of the overall design of the situated 
learning training program; 4) the selection 
and training of the instructor must be 
explicitly considered and planned for in 
situated learning, as in other training 
contexts; 5) the pacing and coordination of 
complex interrelated events in the situated 
learning scenario or script must be carefully 
designed to order to promote both realism and 
learning; and 6) it is necessary to design 
customized scenarios to meet the needs of 
different learners. 

Introduction 

A high-technology training program for 
airline pilots and crew members that has been 
instituted over the past decade exemplifies the 
Situated Learning Model of instructional 
design. Within the airline industry, there is 
increasing concern with pilots' effective 
management of human and technical 
resources to the cockpit, to addition to the 
technical skill traditionally required of 
pilots. This new area of concern is called 
Cr:kpit Resource Management. One outcome 
of this concern has been the development of 
Line-Oriented Flight Traini< g, a program 
that teaches pilots and crew to work together 



on a full-length flight mission so that an 
unexpected series of small problems on a 
flight doesn't escalate into a catastrophe. 
Line-Oriented Flight Training (LOFT) 

synthesizes high-fidelity aircraft 
simulation and high-fidelity aircraft 
simulation and high-fidelity line 
operations simulation to provide 
realistic, dynamic pilot training in a 
simulated line environment. LOFT is 
an augmentation of existing pilot 
training which concentrates upon 
command, leadership, and resource 
management skills. (Lauber & Fouchee, 
1981) 

The LOFT training program provides an 
example of how the Situated Learning Model 
can be adapted to computer-based multimedia 
training. Using complex computer programs, 
LOFT instructors can create almost any 
mechanical problem in the aircraft or 
problems with the navigation aids at the 
destination airport. Line-Oriented Flight 
Training provides pilots with training and 
practice in effective dilemma-handling — the 
moment-by-moment contingencies that 
constitute the conditions of situated actions. 

John Seely Brown and his colleagues at Xerox 
PARC (Brown, 1989; Brown, Collins, & Duguid, 
1989) developed the Situated Learning Model 
to the course of searching to uncover and 
analyze instances of successful learning that 
could provide insights for the design of 
instructional systems. The Situated Learning 
Model is a promising new paradigm for 
instructional design. 

The new training program for airline pilots 
that goes by the name Line-Oriented Flight 
Training (LOFT) is an example of a complex 
multimedia instructional system that 
exemplifies the Situated Learning Model. 
This pilot training program provides an 
example of how sophisticated computer- 
controlled training simulators offer the 
potential for training applications that can 
take advantage of Situated Learning. 
Simulators used for training pilots and other 
professionals who work to technologically 
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sophisticated work environments, such as air 
traffic controllers and nuclear power plant 
operators, are now so realistic that they 
constitute a "virtual" copy of the real-world 
work environment, with one important 
difference: safety Is not a problem. It Is 
Interesting to note that some of the U.S. 
military pilots who served In the recent war 
with Iraq trained entirely in simulators. Now 
that this kind of sophisticated simulator 
technology is available, how can it be used to 
promote optimal training? The Situated 
Learning Model provides some valuable 
guidelines for harnessing this simulator 
technology to promote effective learning 
(Brown, 1989; Brown, etal., 1989). Some 
simulator-based training programs such as 
LOFT that are already to use, exemplify the 
Situated Learning Model. What can be learned 
from this simulator-based training program -- 
- Line-Oriented Flight Training — about the 
design and implementation of training 
programs that conform to the Situated 
Learning Model? This paper will address this 
issue, after first describing the Situated 
Learning Model and the LOFT training 
program in greater detail. 

Situated Learning 

Brown and his colleagues developed the 
Situated Learning Model based on an analysis 
of successful learning programs. Brown found 
one highly successful example in skiing, 
where the learning time has gone down from 
two years to two weeks. In examining this 
example. Brown found a "pervasive use of 
technology," that Included several 
components. The first component was the 
advent of the chair lift which facilitated 
multiple practice. With the chair lift, skiers 
didn't have to expend time and energy 
climbing back up the ski slope after each run, 
so the novice skier could get in far more 
practice runs than ever before. Safety bindings 
and new, better designed ski boots provided 
extra support and scaffolding so that a novice 
skier was less likely to sustain a serious 
Injury and could thus continue practicing 
longer and more safely, reducing the 
possibility of Injury due to inexperience. In 
addition, the Institution of the Graduated 
Length Method (GLM), using gradually longer 
training skis for practice, provided the 
opportunity for wholistic learning and 
Increasingly complex microworlds. The use of 
video replays facilitated reflective practice: 
the learner could review with the ski 
instructor his or her performance, while 
actually watching it on video, thereby gaining 
the advantage of visual feedback, In addition 



to verbal guidance. This use of video replays 
was part of a larger Improvement in skiing 
instruction: better models of coaching. 
Another innovation was to shape or modify 
the environment in order to let the 
environment separate and articulate the 
component skills to be learned. For example, 
In order to learn gliding, you need to be able to 
stop. Can these two skills, gliding and 
stopping, be separated? Yes --- by 
manipulating the environment, Including the 
steepness of the slope and the length of the 
skis (Brown, 1989; Brown, etal., 1989). 

Brown reports that he and his colleagues at 
Xerox PARC identified several examples of 
successful learning, such as the skiing 
example described above. They identified the 
features that these examples shared in 
common and found a set of six useful 
strategies for successful learning: 1) 
Apprenticeship; 2) Collaboration; 3) 
Reflection; 4) Coaching; 5) Multiple practice; 
and 6) Articulation (Brown, 1989; Brown, etal.. 
1989). All of these strategies can be embedded 
in a multimedia simulator-based training 
system. Indeed, these complex and realistic 
simulators appear to be ideally suited for 
facilitating situated learning for Jobs in 
complex technical work environments. 

According to Brown, knowledge is situated; it 
is a product of the activity, context, and 
culture In which it is developed and used. 
Activity and situations are integral to 
cognition and learning. Therefore, this 
knowledge must be learned In context — in the 
actual work setting or a highly realistic 
surrogate of the actual work environment 
(Brown. 1989; Brown, etal.. 1989). 

Airline pilot training In a situated 
environment 

Line-Oriented Flight Training was developed 
In response to Information showing that most 
airplane accidents and Incidents, including 
fatal crashes, resulted from pilot error and 
poor coordination In the cockpit under crisis 
situations. It was found that most accidents 
could have been avoided If the non-flying pilot 
In the cockpit had taken action. The data 
Indicated that the traditional pilot training 
required by regulation, concerning technical 
skills such as how to recover from a stall and 
how to respond when an engine falls, really 
did not really address the kinds of problems 
that resulted In most accidents or incidents. 
LOFT was developed to address these resource 
and crisis management issues through 
training (Ruffell Smith. 1979; Stockton, 1988; 
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Coffman, 1989). 

The LOFT training program provides practice 
in team building and crisis management 
(Lauber & Fouchee, 1981). Pilots learn to work 
together effectively so that an unexpected 
series of small problems on a flight doesn't 
escalate into a catastrophe. This training 
program is designed to promote a work 
environment where there is a high level of 
technical skill coupled with a high regard for 
people. Captains are urged to take charge in 
their cockpits but at the same time they are 
encouraged to listen to what other crew 
members have to offer (Lauber & Fouchee, 
1981; Stockton. 1988; Cofrman, 1989). 



In Line-Oriented Flight Training, the goal is to 
replicate, as closely as possible, the actual 
flight environment, complete with delays, 
adverse weather, even unruly passengers 
(Lauber & Fouchee. 1981). Evidence suggests 
that crews often get into trouble when they get 
distracted by the unexpected. They sometimes 
forget procedures designed to ensure safe flight 
operations. The premise underlying LOFT is 
that by having flight crews practice handling 
those kinds of distracting situations in 
sophisticated simulators, they will be 
prepared for real-world emergencies. Using 
complex computer programs. LOFT 
Instructors can create almost any mechanical 
problem in the aircraft or problems with the 
navigation aids at the destination airport. 

On a typical LOFT simulated flight, more than 
one thing goes wrong. Rather than 
catastrophic problems, the LOFT crew 
encounters a series of small difficulties that 
demand teamwork and coordination. In the 
Line-Orieted Flight Training simulations, 
there is no Instructor intervention; the crew is 
allowed to follow its own course wherever it 
may lead. The consequences of crew decisions 
and actions during a LOFT scenario mount up 
and Impact the remainder of the trip in a 
realistic manner. In the beat LOFT programs, 
a television camera records the flight crew, 
pilot and co-pilots, at work. After the 
simulated flight is over, an instructor sits 
down with the crew to critique the pilots' 
performance. These debriefing sessions are 
where the learning really occurs. Ideally, 
there is no punitive aspect to the training. The 
emphasis is on encouraging pilots to change 
their habits, rather than on forcing them to do 
so (Stockton, 1988; Coffman. 1989). 

Effective resource management Involves the 



management of human error. According to 
Lauber and Fouchee (1981), " Effective resource 
management recognizes that under some 
circumstances, such as high -workload 
situations, human error is likely; steps must 
be taken to reduce the probability of error." 
Errors, when they occur, must be detected and 
corrected, in order to minimize the 
probability of adverse impact upon the 
overall safety of the operation. Just as it is 
necessary to practice landing skills In order to 
gain and maintain aircraft-handling 
proficiency, it is necessary to practice human- 
error-management skills. Training in 
effective error management requires the 
presence of errors or error-inducing 
situations. 

The LOFT training program for flight crews Is 
an example of situated learning in a 
sophisticated multimedia training 
environment. All six of the critical situated 
learning components — Apprenticeship; 
Collaboration; Reflection; Coaching; Multiple 
practice; A iculation — are present in the 
LOFT trainliig program. Within the simulated 
flight, the environmental conditions are 
controlled, modified, and articulated by the 
Instructor to simulate increasingly complex 
conditions. The learning environment is 
contextually rich and highly realistic. 
Apprenticeship is present since the instructor 
decides on what array of interlocking 
problems to present on each simulated flight. 
The pilots must gain experience with different 
sets of problems In order to build the skills 
neccesary for teamwork and coordination. 
And they must learn to solve problems for 
themselves: there is no instructor 
intervention during the simulated flights; 
LOFT crews are allowed to follow their own 
course wherever it may lead. Reflection is 
scheduled Into the training after the simulated 
flight is over, when an Instructor sits down 
with the crew to critique the flight crew's 
performance. This involves coaching from 
the instructor as well. The simulation 
provides the opportunity for multiple practice. 
Including practice where different factors are 
both articulated and combined together 
realistically. The emphasis on collaboration 
between workers in the cockpit — a key goal of 
the LOFT training --■ makes this example 
highly compatible with the Situated Learning 
Model. 

LOFT simulations provide a vehicle for 
demonstrating the Importance of effective 
cockpit resource management, and they 
provided crews with vivid demonstrations of 
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operational complications that can result 
when resources are ineffectively or 
Inappropriately utilized. However, LOFT is 
not a replacement for maneuver-oriented 
flight training. An essential prerequisite of 
effective cockpit management is a highly 
skilled, highly knowledgeable pilot. 
Proficiency in manual control of the aircraft 
and in the operation of its systems is primary 
— without it, no amount of management, 
command, or leadership training will produce 
a safe, proficient, and effective pilot (Lauber & 
Fouchee, 1981). 

Stories 

The Line-Oriented Flight Training program 
for pilots emphasizes stories: stories of real 
disasters and simulated stories or scenarios of 
crisis situations that represent all the possible 
kinds of technical and human problems that a 
crew might encounter in the "real world." 
This is another fundamental component of 
situated learning. 

According to the Situated Learning Model, In 
order to make the training memorable, it must 
be embedded In a story, a scenario. Brown 
(1989) emphasizes the importance of stories 
for effective learning. According to Brown, 
narratives play a vital role In the transfer and 
Integration of information and discoveries. 
And stories help people keep track of their 
discoveries. As stories are shared, fragments 
of other stories are melded together with them 
that enhance the meaning of the original 
stories, as well as transferring information 
about the discoveries to others in the 
workplace. According to Brown, 
troubleshooting is basically storymaklng. 
Swapping stories essentially means 
bootstrapping on each other's experience. By 
exchanging stories, we "bootstrap" on a 
community of experience, our own experience 
and that of co-workers. Sophisticated 
simulations, designed to teach people how to 
make sense out of complex, unclear data to 
solve problems, are a form of storytelling, or 
perhaps scenario building. Brown (1989) 
explains that stories are anchored in episodes. 
Workers can refer back to stories or scenarios 
in their training in attempting to make 
effective decisions in real-life work 
situations. 

This is similar to a model put forward by 
Brenda Laurel whose background is in the 
theater. Laurel (1989) articulates a metaphor 
of computer-as-theater. She comments, 
"millennia of dramatic theory and practice 
have been devoted to an end that is 



remarkably similar to that of human- 
computer interaction design; namely, creating 
artificial realities in which the potential for 
action is cognitively, emotionally and 
aesthetically enhanced." According to Laurel, 

It is not enough to imitate life. Drama 
presents a methodology for designing 
worlds that are predisposed to enable 
significant and arresting kinds of 
actions — - where characters make 
choices with clear causal connections to 
outcomes, where larger forces like 
ethics, fate or serendipity form 
constellations of meaning that are only 
rarely afforded by the real world. 
Dramatically constructed worlds are 
controlled experiments, where the 
irrelevant is pruned away and the bare 
bones of human choice and situation 
are revealed through significant action. 
The predispositions of such worlds are 
embodied in the traits of their 
characters and the array of situations 
and forces embedded in their contexts. 
If we can make such worlds Interactive, 
where a user's choices and actions can 
flow through the dramatic lens, then we 
will enable an exercise of the 
imagination, intellect and spirit that is 
of an entirely new order." 

Laurel's vision of dramatically constructed 
worlds is highly relevant to the design of 
training environments in accordance with the 
Situated Learning Model. The Line-Oriented 
Flight Training scenarios are planned and 
scripted to be realistic in terms of the pacing 
and interaction of distracting events, but also 
so that these events are dramatically combined 
and compressed to promote learning in an 
environment that improves upon the natural 
environment of actual flights for the purpose of 
learning. 

Implications for the design of situated 
learning-based training 

The LOFT program has several important 
implications for the design of situated 
learning- based training. One interesting 
aspect of Line-Oriented Flight Training for 
flight crews is that it is situated learning for 
teams rather than individuals. The 
implication here is that the debriefing process 
must be designed to provide two kinds of 
feedback: feedback for the team and feedback 
for the individual. Lauber and Fouchee (1981) 
describe how this is achieved In LOFT: "During 
debriefing, both total crew performance and 
individual performances should be openly 
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discussed and assessed by the instructor. 
Critical assessment of an individual can be 
mentioned in the presence of the full crew, but 
remedial details should be handled privately." 

A second implication is the importance of 
differentiating between training and testing in 
the situated learning environment. LOFT is 
designed as a learning experience in which 
errors will probably be made, not a checking 
program in which errors are unacceptable. 
Lauber and Fouchee recommend that for 
maximum effectiveness, LOFT must be 
perceived as pure training, not testing or 
checking performance, by both crew members 
and instructors. The instructor must set the 
tone, emphasizing during the preflight briefing 
that LOFT *s designed purely as a learning 
experience, with no job testing strings 
attached, although additional training may be 
prescibed if the instructor determines it to be 
necessary. LOFT is "learning through 
experience, which includes making mistakes 
and errors. To keep minds open, to benefit 
most from the experience, it is essential that 
LOFT be entered Into with a feeling of freedom, 
openness, and enthusiasm (Lauber & Fouchee, 
1981)." Lauber and Fouchee emphasize that 
when dealing with issues such as crew 
coordination, command, leadership, and 
resource management. Insight into individual 
limitations and weaknesses is an important 
component of learning and training. And the 
instructor must be tactful to avoid the 
appearance of checking rather than training. 
It is essential to address these factors so that 
concern with performance evaluation will not 
detract from training (Lauber & Fouchee, 1981). 

A third Implication for the design of situated 
learning-based training concerns the role of 
the instructor in a situated learning 
environment. During Line -Oriented Flight 
Training, the instructor's role is one of 
communicator, observer, and moderator. 
During the simulated flight phase of LOFT, the 
instructor in not an 'instructor' in the 
traditional sense. Instead, the instructor is 
"the 'coordinator' or manager of the flight, 
using appropriate radio calls or responses to 
direct the flight along the desired path; he must 
be prepared to accept and manage alternate 
courses of action that the crew may wish to 
follow (Lauber & Fouchee, 1981)." These 
responsibilities require a considerable amount 
of creativity and restraint on the part of the 
Instructor. The Instructor should resist the 
temptation to 'instruct' or intrude into the 
situation, thereby destroying the sense of 
realism that is vital to LOFT training. And the 



instructor must not offer any hints, or In any 
other way remind the LOFT crews that they are 
in a simulator training session. The 
instructor's role is to manage the training 
situation, not to 'teach' right solutions, or to 
'test' the trainees (Lauber & Fouchee, 1981). 

During the debriefing after the LOFT flight, the 
instructor must guide the discussion to explore 
the full range of potential approaches to 
problems that emerge during the simulated 
flight. The Instructor must make sure that key 
topics relating to cockpit management, such as 
crew management, crew coordination, and 
crew communications are discussed. The 
discussion should also cover the use of Airport 
Traffic Control and company 
communications; manuals, charts, and other 
software; the use of other crewmembers; and 
the use of the autopilot, autothrottle, and other 
potential workload-reducing d-wlces (Lauber & 
Fouchee, 1981). 

During the debrief *ng, the instructor should do 
everything possible to foster self-analysis 
while at the same time keeping it at a 
constructive level. In his role as moderator, 
the Instructor can guide the discussion to areas 
relevant to the completed LOFT flight that need 
attention. The instructor should raise 
questions about certain procedures, decisions, 
and mistakes that transpired during the LOFT 
flight. However the instructor should avoid 
'lectures' about what is light and what Is wrong 
unless absolutely necessary. And the 
Instructor should try to avoid embarrassing 
members of the crew (Lauber & Fouchee, 1981). 

A fourth implication of the LOFT training 
program for the design of situated learning 
training programs is the selection and training 
of the instructor. Lauber and Fouchee 
emphasize: "LOFT instructors should receive 
rigorous training in the philosophy, 
principles, and conduct of LOFT. " Areas that 
need special attention include: learning how to 
maintain a realistic atmosphere and avoid 
intrusiveness during a simulated flight, 
effective observation skills, script writing and 
scenario design, pacing and coordination, 
creativity and adaptability, and interpersonal 
skills. The LOFT instructor must also have an 
in-depth understanding of the dimensions of 
performance related to the exercise of 
command responsibilities, crew coordination, 
leadership, decisiveness, and interpersonal 
communication. And the instructor must have 
a thorough familiarity with aircraft systems, 
performance, and procedures, and they should 
be able to assess certain intangible assets such 
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as flyhig skill and "smoothness." Knowledge 
of, and compliance with, Federal Aviation 
Regulations and Airport Traffic Control 
procedures is another area that Instructors 
must know in detail (Lauber & Fouchee, 1981). 

A fifth implication is the need for careful 
design in developing a complex multimedia 
situated learning training environment during 
both scenario script development and LOFT 
flight Implementation so that pacing and 
coordination of interrelated events are 
designed effectively to promote both realism 
and learning. Abnormal and emergency 
procedures and situations must be paced and 
introduced carefully and realistically. And it 
is essential for the LOFT instructor to 
understand the ways In which he can adapt the 
scenario to handle unforeseen crew actions 
(e.e., rendering navigational aids inoperative, 
closing airports because of adverse weather, 
etc.). The creative nature of this aspect of LOFT 
operation must be clearly understood (Lauber & 
Fouchee, 1981). 

A sixth implication of the LOFT program for 
the designers of situated learning is the need to 
consider and design customized scenarios to 
meet the needs of different learners. Pilots 
who usually fly short distance routes will need 
very different scenarios than those flying long, 
nonstop routes. Much of the realism of LOFT is 
lost if the scenarios are not consistent with a 
carrier's route structure or if the crew is unable 
to use actual charts, manuals, and other 
materials. And the design and development of 
a LOFT program should be guided by a 
consideration of the skills required of an 
individual pilot, as well as the skills necessary 
for a fully integrated flight crew, such as crew 
coordination and cockpit resouce 
management. A well-designed LOFT scenario 
should exercise both sets of skills. LOFT is also 
a good vehicle for providing experience with 
problems in aviation operations such as 
distraction, complacency, forgetting, and 
failure of information transfer. All LOFT 
scenarios and flight segments should be 
designed on the basis of a formal and detailed 
statement of specific objectives and desired end 
products (Lauber & Fouchee, 1981). 

In conclusion, LOFT is a highly complex 
simulator-based training program that 
exemplifies all six of the components that 
characterize situated learning: 
Apprenticeship; Collaboration; Reflection; 
Coaching; Multiple practice; and Articulation. 
In addition, LOFT emphasizes stories: stories 
of real disasters and simulated stories or 
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scenarios of crisis situations; stories are 
another fundamental component of situated 
learning. 

As an example of situated learning In a highly 
realistic surrogate of the actual work 
environment, the LOFT program has several 
Important implications for designers seeking 
to implement training in accordance with the 
Situated Learning Model: 

1) the debriefing process for a team of 
learners, rather than an individual 
learner, must be designed to provide 
feedback for both the team and the 
Individual; 

2) it is essential to differentiate between 
training and testing in the situated 
learning environment and to make sure 
that this distinction is clear to both the 
instructor and the learners; 

3) the role of the instructor in a situated 
learning environment must be carefully 
designed as part of the overall design of the 
situated learning training program; 

4) the selection and training of instructors 
must be explicitly considered, and planned 
for, In situated learning, as in other 
training contexts; 

5) the pacing and coordination of complex 
interrelated events in the situated learning 
scenario or script must be carefully 
designed in order to promote both realism 
and learning; and 

6) it may be necessary to design customized 
scenarios to meet the needs of different 
learners. 

This pilot training program provides an 
example of how sophisticated computer- 
controlled training simulators offer the 
potential for training applications that take 
advantage of the Situated Learning approach to 
Instructional design. 
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THE DEVELOPMENT OP AN INTERACTIVE SUPPORT SYSTEM SHELL 
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Abstract 

Interactive computer support systems have 
been developed for student learning during the 
past two decades on an individual basis. These 
systems have offered learners the opportunity 
to evaluate current self-understanding and to 
supplement personal knowledge with 
additional relevant information. 
Instructional professionals were required to 
design, write programming code and enter data 
into the application program. The use of an 
interactive support system shell affords the 
instructor the opportunity to utilize the 
instructional software logic and structure 
while the content data is altered by the 
instructor to fit the needs of the learner. 

A support system shell has been designed to 
allow the learner to access multiple choice 
questions, to be presented with the correct 
answer and a page reference when an incorrect 
response is given, to have access to an online 
content area dictionary, and to receive an 
evaluation of learning report at the end of a 
review session. The instructor is given the 
resources to enter the multiple choice 
questions, to enter the terms and definitions 
for a content aTea dictionary, to alter the 
Instructional presentation sequence, to modify 
the questioning pattern, and to gather learner 
performance data. 

The design of the support system shell and a 
working model will be presented and 
demonstrated at the conference. 

Introduction 

During the past decade development and 
research of interactive computer-assisted 
Instructional programs he. 'e been conducted 
across various content areas using a variety of 
Instructional strategies and techniques (Park 
& Tennyson, 1986; Tennyson & Cocchiarella, 
1986; Goetzfried & Hannafin, 1985a; Kulik, 
Schwalb & Kulik, 1982, Tennyson & Park, 
1980; Hartley, 1978). Within each of these 
studies, researchers investigated a particular 
instructional strategy applied to a specific 
content area. The results of these studies have 
demonstrated both significant and non- 
significant results; however, when the results 
of a particular study were shown to be 
significant, the flexibility of the program was 
limited to the embedded content data of the 



study. An issue which had not been considered 
in the studies was the importance of using the 
software application program by educators in 
alternative content areas. Similar to the 
development of expert-system shells, which 
allow for a specific inference engine to access 
varied knowledge bases, instructional support 
system shells need to be created from effective 
CAI studies so that the instructional strategy is 
maintained within the application program 
and the content area can be installed by each 
instructor. In viewing CAI program 
development in this manner, effective research 
studies can be taken from Journals and 
professional magazines and placed into the 
hands of variety of professional educators. 

If an effective instructional strategy has been 
developed for reading comprehension, 
instructional review, diagnostic testing or 
another domain using a specific content area 
then the instructional professional should 
have the opportunity to alter the content area 
to one distinct from that used in the research 
software. The sectioning of the program code 
so that the content data Is not embedded within 
instructional strategy will help to facilitate the 
application of effective research studies 
throughout the educational community and 
across varied content areas. The development 
of an interactive support system shell affords 
the Instructor the opportunity to utilize the 
support system shell and to alter the content 
data for the needs of a particular learner group 
and content area. 

An Application Developed 

Several issues need to be considered in the 
development of interactive support system 
shells, namely, is there a practical need for the 
software application and the learning strategy, 
are the research results significant for the 
educational strategy or technique, and is it 
feasible to separate the program logic from the 
content data. Research studies (Robyler, 1985; 
Ross, 1984; Hannafin, 1984) have considered 
the first two of these points; however, little has 
been stated In the body of research literature 
concerning the application of a particular 
study to alternative content areas. Most 
presentations and/ or publications focus on the 
significance of the research study, and 
illustrate segments of the application program 
concerning the structure and the strategy of the 
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software. For those with design and/or 
programming experience this is of great 
benefit, but this situation leaves many with the 
task of programming In their own content data 
or matching content needs exactly to that of the 
research study in order to utilize the results in 
their own particular area of Interest. 

It seems reasonable to assume that in the 
future, programming professionals on the 
research team will consider the structuring and 
access of the program code separate from the 
creation and access of the content data. The use 
of external files and databases can be designed 
to facilitate the process of separation of 
program logic and Instructional content. This 
would allow for the utilization of content data 
from various curriculum areas with 
Instructional algorithms and strategies from 
research studies demonstrating significant 
results. 

The Re view Program 

During the 1989-90 academic year a computer- 
assisted instructional review program 
(Mlchellnl, 1990b) was developed to assist 
sophomore college students In learning 
computer concepts. The program was designed 
to supplement the material covered In the 
weekly lectures of an Introductory computer 
course within the School of Business. The 
review program consisted of approximately 
800 multiple-choice questions which were 
subdivided into 18 chapters dealing with 
specific aspects of computer technology. 

Each student was given a distinct logon 
Identification name and unrestricted access to 
the review program during normal computer 
lab hours. Upon entry into the review program 
the student was presented with an introductory 
screen explaining how to use the program. 
Students selected a particular chapter for 
review and the program would proceed to 
present multiple choice questions pertaining to 
that specific chapter. A question was presented 
on the terminal screen; the student was asked 
to select the correct response; in the event of an 
Incorrect response by the student, the student 
was given another opportunity to select the 
correct response. Some form of recognition 
was given to the student after answering each 
question -- acknowledgement of the student's 
correct response, acknowledgement of an 
incorrect response by the student followed by 
the correct response and a page reference to 
Information related to that specific question. 
Student data was collected by the program 
reflecting each individual question and 
cumulative totals for that particular chapter. 



At the end of each review chapter the student 
was given a performance evaluation with 
respect to that specific chapter. 

Chapter questions were presented to each 
student in random order during a given review 
session, and a chapter review session was based 
upon a fixed number of questions, a learner 
determined number of questions, or a last four 
question response pattern algorithm. The 
flexibility of random order question 
presentation allowed a student to review work 
on separate occasions and be presented with a 
distinct pattern of questions. The random 
order also provided instructional security in 
the event that two students sitting at adjacent 
computers would not receive the questions in a 
similar pattern. 

The review program was written in the BASIC 
programming language; run on a Novell 
network using 48 IBM PCs with 640K RAM. The 
review program was down loaded to each 
individual PC and the program accessed the 
system hard disk to retrieve chapter questions 
and to maintain student records. Students 
were not aware of the fact that response data 
was being collected; however, they were aware 
of the correctness of each response and of their 
overall performance within a given chapter. 
An advantage to Implementing the review 
program using a network configuration was the 
ease of maintaining the program and the 
question data, the ease of collecting student 
response data, the flexibility to handle 
multiple users on the same system, and size of 
the data files could be expanded without user 
disk swapping inconvenience. 

The review program (Mlchellnl, 1990a) was 
expanded during the spring and summer of 
1990 to Include an online dictionary of 
computer terms. The dictionary could be 
accessed by the user during the review session 
for the definition of terms related to a 
particular chapter question or a specific 
computer concept. The dictionary of computer 
terms was created by the instructional 
developer prior to program use by the learner 
and expanded or maintained by the 
Instructional developer whenever necessary. 

Modification toward an instructional shell 

During the Fall of 1990 a reevaluation of the 
review program and the online dictionary of 
terms was made in an attempt to generalize the 
software for use by professionals in other 
academic areas. Attention focused on two 
distinct areas of the review software: the entry 
by the instructor of alternative content areas 
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for the multiple-choice questions, and the 
creation and use of an online content area 
dictionary in conjunction with the multiple- 
choice questions. Since the review program 
was constructed using external data files, the 
content data was already separated from the 
program structure itself. It seemed reasonable 
to assume that the program could be modified 
to allow professionals from other educational 
areas the opportunity to supply their own 
dictionary and multiple-choice questions 
while utilizing the program structure. Tills was 
accomplished by creating a variable that would 
allow for an indirect reference to the external 
file containing the relevant data - either one of 
the multiple-choice questions or a content area 
dictionary definition. 

A program written in the BASIC programming 
language would include the following 
statement to open an external file containing 
question data for input into the program: 

290 OPEN "MEMORY.DAT" AS #1 FOR INPUT 

This statement is identifying the external file 
that the program must access in order to 
retrieve a multiple-choice question for use in 
the program. As written this external file 
name is locked into the program by the 
programmer and cannot be changed during 
runtime; the program code needs to be altered 
by a programmer in order for a different data 
file to be accessed. 

Using an indirect reference technique the 
program is altered to contain the following 
code, 

190 INPUT "Enter the name. ", CHAP1 1$ 

• • • 

300 OPEN CHAP1 1$ AS #1 FOR INPUT 

which allows the instructor the opportunity to 
name the external file and to reference the 
external file by one's own eight character 
identifiers (B:TELECOMM.DAT). The program 
logic is still built around a fixed identifier, 
CHAP 11$; however, the instructor is only 
aware of the variable name 
(BTELECOMM.DAT) entered by the instructor, 
and the computer system stores the file under 
this name. This procedure allows the 
instructor to create and maintain the external 
files. The programmer has used a series of 
fixed identifiers (CHAP1 1$) as in line 300 of the 
program, but what the the instructor has done 
is assign that identifier an external file name. 
What the instructor experiences is the system 
asking for a chapter file identification; once 
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the identification is given the program 
maintains the appropriate references. A 
positive aspect of this type of file identification 
is that the Instructor can vary file placement 
according to the characteristics of the local 
system, namely, files can be stored or various 
disk drives (A, B:, C:, or in a network setting). 

The use of this indirect naming technique is 
not without some restrictions or limitations. 
The review program assigns variable names 
(CHAP11$, etc.) to hold instructor supplied 
chapter and data dictionary identifiers. Since 
the programmer needs to determine identifiers 
before runtime,' the program is limited to the 
number of chapters assigned by the program 
code, and by the fact that the program needs to 
create a special external data file to store the 
chapter variable names assigned by the 
instructor. All the chapter variable file names 
need to be stored and maintained between 
sessions of program use, otherwise, those 
names would be lost to the program and data 
access would not be possible. A chapter 
variable file Is created to maintain the chapter 
naming references of the program for the future 
use of the program. 

Results and implications 

The initial Interactive review program has 
been well received by the students taking the 
computer concepts course. In each of the past 
four semesters students have indicated on 
teacher evaluation forms and verbally that 
they had profited from the review program. 
This evaluation was in respect to both 
improved personal understanding of the 
computer concepts, and In test preparation 
during the semesters. The program was made 
available to all students registered for the 
course and an analysis of performance with a 
control group has not been conducted at present 
time. 

The results of modified program (programming 
shell version) has demonstrated that the 
structural aspects of a program can be 
separated from the content data so that non- 
programming professionals can utilize the 
program for their own content area. The 
programming shell version is currently being 
used to create new external content data file for 
use with a new text. Using a shell version of the 
program offers instructors increased 
flexibility In molding the content data to one s 
own needs and to the requirements of the 
learner. What is of importance here is the fact 
that the local instructor has the power to 
determine the content data while the 
researcher offers a proven learning strategy 
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Introduction 

Interactive instruction (computer-assisted 
instruction, interactive video) is often an 
extension of programmed instruction 
developed in the behavioral psychology era. 
The emphasis is on shaping the behavior of 
students. In this programmed-lnstruction- 
based courseware the primary emphasis is 
well-designed stimuli presented as displays of 
text or graphics. Often these displays enable 
only a limited number of interactions between 
the courseware and students. The most 
frequently seen Interaction is presenting text 
and graphics, asking a question, evaluating 
the student's response, providing feedback, and 
branching to the next display (Merrill. 1985). 

Most CBT authoring systems have a frame- 
based architecture. The primary element is a 
display (screen) of information consisting of 
graphics and text. The learner is asked a 
question, usually consisting of one of the 
standard question forms of multiple-choice, 
short answer, true false, matching. Depending 
on the learner answer, another frame of 
information is presented. This alternative 
path procedure is called branching. Frame- 
based architecture requires that each frame be 
individually authored and stored in a data base 
that resembles a file cabinet of displays that 
can be accessed in various orders depending on 
the branching structure. 

Many nonlnstructional computer applications 
have a different architecture, that is, an 
algorithm plus data. An algorithm is a 
sequence of computations that can be repeated 
over and over with different data. A frame- 
based CAI architecture may be considered as 
one kind of algorithm, but a very limited 
algorithm consisting of branching from one 
display to another. Subject matter content, the 
knowledge and skills to be learned, can be 
separated from the instructional interaction 
with the learner by which these content 
elements are learned. The instructional 
interactions are algorithms for interacting 
with the learner. The subject matter content is 
data that is used by these instructional 
algorithms. 

Instructional transactions 1 are instructional 



algorithms, patterns of learner interactions 
(usually far more complex than a single display 
and a single response) which have been 
designed to enable the learner to acquire a 
certain kind of knowledge or skill. Different 
kinds of knowledge and skill would require 
different kinds of transactions. The necessary 
set of these instructional transactions are 
designed and programmed once, like other 
applications such as spread sheets and word 
processors. These instructional programs are 
called instructional transaction shells. These 
transaction shells can then be used with 
different content topics as long as these topics 
are of a similar kind of knowledge or skill. 

Authoring by way of instructional transaction 
shells consists of selecting those patterns of 
interactions which are appropriate for a given 
topic and merely supplying the subject matter 
content in a form that can be used by the 
transaction shell. There is no need to 
determine every display; to determine a 
branching structure; to select what kind of 
questions to use, to specify answer processing. 
Once the transaction shells have been 
developed they can be used over and over again 
with no need for extensive instructional design 
or programming. The cost savings of this 
algorithm plus data approach to the 
development of courseware is many times more 
efficient than a frame-based approach. A one 
hour lesson that may require 200 or more 
hours of development using a frame-based 
approach can be developed In 20 or 30 hours 
using a transaction shell approach. 
Furthermore, the frame-based approach may 
limit the interactions to answering questions 
whereas a transaction can involve interactive 
environments which enable the learner to 
explore the subject matter, manipulate 
simulations of devices, and other more 
effective and more complex Interactions that 
are impossible or impractical to build using 
frame-based systems. 

Instructional Transaction Shells 

An instructional transaction is a mutual, 
dynamic, real-time give-and-take between an 
instructional system and a student in which 
there is an exchange of information. It Is the 
complete sequence of presentations and 
reactions necessary for the student to acquire a 
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specific type of instructional goal. It requires 
active mental effort by the student. Its 
effectiveness Is determined by the match 
between the nature of the student's Interaction 
and resulting mental processing with the type 
of task and subject matter content to be 
learned. 

We subscribe to the notion that learners more 
easily store cognitive representations of 
knowledge and skill bundled into interrelated 
knowledge structures rather than unrelated 
bits and pieces of information. All of the 
knowledge and skill required to engage In some 
complex activity -- such as solving linear 
equations, driving a car, using an electronic 
spread sheet -- are highly interrelated and 
constitute a mental model. This complex 
activity, enabled by a mental model, is called 
an enterprise (See Gagne & Merrill, 1990). 

We distinguish several other terms: A 
transaction shell 2 identifies the interactions, 
knowledge representation and parameters 
needed for a given class or family of 
transactions. When a transaction shell Is 
instantiated with a particular subject matter 
and with particular values for its parameters, 
it is called a transaction instance. Both a 
transaction shell and a transaction instance 
are pieces of computer code that, when 
delivered to a student via an appropriate 
delivery system, cause a transaction or set of 
transactions to occur. We are not always 
careful to distinguish the computer objects 
which cause a transaction to occur from the 
transaction which is the actual interaction 
with the student. 

A transaction class is a set of similar 
transaction shells which have similar 
interaction requirements and similar 
knowledge representation requirements. A 
transaction family is all of the transactions 
necessary to enable a learner to acquire all of 
the knowledge and skill required to engage in a 
particular enterprise. An enterprise 
transaction is a higher level transaction which 
accomplishes two purposes: first, it functions 
as a transaction manager, providing the 
overall direction of the execution of the 
individual transaction instances in the 
transaction family. Second, it provides for an 
Integration of the learning facilitated by the 
individual transactions in the transaction 
family. 

A transaction shell consists of four primary 
components: interactions and an interaction 
manager which causes the transaction to occur; 



instructional parameters which enable the 
instruction to be customized for a given learner 
population, learning task, and environmental 
situation; a knowledge base containing a 
structural representation of all the knowledge 
to be taught; and a resource data base 
containing mediated representations of the 
knowledge to be taught. A transaction shell 
has three authoring systems: a transaction 
configuration system, a knowledge acquisition 
system, and resource editors. The knowledge 
acquisition system enables a subject matter 
expert to structure the knowledge to be taught. 
The resource editors enable the creation of 
mediated representations of the knowledge. 
The transaction configuration system enables 
the designer to provide values for a wide range 
of Instructional parameters. 

Transaction shells incorporate intelligence 
about instructional design In several ways. 
First, the functions and methods of each 
transaction shell enable the type of 
interactions most appropriate for acquiring a 
given type of knowledge. The designer does not 
need to reinvent appropriate instructional 
designs for every application. Second, the 

knowledge base includes a syntax 34 for 
knowledge representation that not only 
enables transactions to use this knowledge, but 
assures that the knowledge included is 
complete and consistent. Third, intelligence in 
the knowledge acquisition system enables 
subject matter experts to supply the necessary 
knowledge without knowing the formal syntax 
of the knowledge representation system. 
Fourth, the parameters of each transaction 
shell identify those ways that these 
interactions can vary for different learner 
populations and different tasks. Fifth, 
intelligence in the transaction configuration 
system contains instructional design rules 
relating learner and task attributes to various 
values on the instructional parameters. Thus, 
the Instructional designer needs to supply only 
descriptive information about the learners and 
the task, the configuration system can select a 
pattern of Instructional parameter values 
consistent with this information. However, 
the configuration system is merely a guide, a 
designer has access to all of the parameters and 
can adjust the value of groups of parameters or 
individual parameters to more adequately 
fine-tune a particular transaction instance. 

Functions of Instructional Transactions 

The Instructional design prescriptions of first 
generation instructional design are 
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Figure 1 Components of an Instructional Transaction Shell 



characterized as "best case" prescriptions. In 
our own previous work 5 we have identified 
different prescriptions for each of several 
performance-content outcomes. These 
prescriptions identify values for a number of 
variables which characterize the best 
Instructional strategy for each of the possible 
outcomes. These prescriptions, however, do 
not Identify the range of values for the many 
parameters which characterize each 
prescription, nor do they Indicate conditions 
for which these parameters should assume 
different values. In other words, each outcome 
classification has a "best" case prescription 
and deviations from this prescription are left 
to the individual Instructional designer. 
Transaction shells do not merely represent a 
best case. By changing its parameter values it 
can be configured in many ways to represent a 
complete range of Instructional Interactions. 

Several Instructional functions are necessary 
in order for an instructional transaction to 
successfully interact with a learner. All 
instructional transactions, regardless of the 
type of knowledge or skill taught, must be 
capable of performing these functions. The 
specific parameters that are necessary for a 
given type of transaction to accomplish these 
functions will differ from one class of 
transaction to another. In fact, the difference 
in the way these functions are accomplished by 
different classes of transactions is one of the 



characteristics that distinguish one class of 
transaction from another. 

Each function is accomplished via several 
methods that are specific computer programs 
that enable the function to be accomplished. 
These methods require values on a number of 
instructional parameters 6 . These parameter 
values determine exactly how a given method is 
applied in a given transaction instance. The 
interactions enabled by a given transaction 
can exhibit a considerable variance depending 
on the values assigned to the parameters which 
constrain its methods. Instructional design 
via instructional transaction shells consists of 
selecting parameter values appropriate for a 
given learner population and particular 
learning task. These parameter values then 
enable the methods of each function to carry 
out this function in a way consistent with the 
requirements of a given learning situation. 

All instructional transactions must include 
the following functions: knowledge selection, 
knowledge sequence, instructional 
management, and instructional enactment. 
Select Knowledge 

From all the knowledge associated with a given 
transaction instance, the knowledge selection 
function determines that part which will be 
taught during a particular enactment of the 
transaction. Associated with each of the 
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knowledge frames for a given instance of a 
transaction is a resource data base containing 
the mediated representation which will be 
presented to the student by the transaction. 
Each of the frames in the knowledge base may 
include a large number of components: parts, 
steps, or events. Each of these knowledge 
frames may be implemented by several 
different mediations In the instructional 
resource data base. The amount of available 
knowledge often exceeds that which needs to be 
presented during a given enactment of the 
transaction. When a transaction is sent a 
message to do its Job. the first parameters It 
needs are those which tell it of all the 
knowledge that is available, which specific 
knowledge elements are to be Included during 
this enactment of the transaction. 

Sequence Knowledge 

The knowledge sequence function determines 
which of the selected knowledge elements Is 
presented next. Whenever the amount of 
knowledge to be included in a given enactment 
of a transaction exceeds that which should be 
presented 7 simultaneously, then an 
instructional transaction requires sequence 
parameters to indicate how this knowledge 
should be partitioned and sequenced. 
Knowledge acquisition is facilitated If the 
knowledge is partitioned into mind-size pieces; 
on the other hand knowledge assessment often 
requires the learner to Interact with the 
knowledge as a whole. A given instructional 
transaction, regardless of the type of 
knowledge taught, should be able to Invoke a 
variety of Instructional sequences. 

Manage Interactions 

The Instructional management function 
determines how the student will interact with 
the selected and sequenced knowledge. 
Instructional management is accomplished by 
the selection of an Instructional strategy. An 
instructional strategy Is a sequence of 
interaction modes, each of which knows how 
to either overview information, present 
information, facilitate the students' practice of 
the skills promoted, or assess the students' 
knowledge and skill. The management 
function also determines when a learner 
should move to the next interaction mode in 
the strategy. Instructional strategies can vary 
from providing information to promoting 
mastery of the knowledge and skills involved 
in the transaction. The type and sequence of 
interaction modes varies from one strategy to 
another. A given Instructional transaction, 
regardless of the type of knowledge taught, 
should be able to Invoke a variety of 
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instructional strategies. 
Enact Interactions 

The Instructional enactment function 
determines how each Interaction mode in a 
given strategy carries out its responsibility. 
The enactment function determines the role a 
given interaction will play whether presenting 
information, enabling practice, or assessing a 
student. The enactment function specifies how 
the interaction presents Information, 
constrains learner responses, and/or reacts to 
the learners responses. The enactment 
function also determines how each interaction 
is adjusted to provide the type of interaction 
most appropriate to a given student and subject 
matter. A given interaction mode, regardless 
of the type of knowledge taught, should be able 
to modify the nature of Its interaction with the 
student in a variety of ways. 

Classes of Instructional Transactions 

We propose that instructional transactions can 
be grouped into a limited number of classes. 
The nature of the Interactions for a given class 
of transaction depends on the type of 
knowledge structure(s) that the transaction 
seeks to promote and the learner performance 
enabled by the transaction. We assume that 
different knowledge structures require 
different types of Instructional transactions. 
We also assume that different transactions 
promote the acquisition of different types of 
learner performance. 

We have identified three primary classes of 
transactions', component transactions, 
abstraction transactions, and association 
transactions corresponding to the three forms 
of elaboration In the knowledge 
representation. The knowledge frames 
required for a given transaction are called a 
transaction jrame set The transaction frame 
set for a component transaction Is a single 
knowledge frame and the components which 
comprise this frame. The transaction frame 
set for an abstraction transaction is at least a 
class frame and two or more instance frames 
from an abstraction hierarchy. The 
transaction frame set for an association 
transaction is two or more associated frames 
from the Elaborated Frame Network. 

Component transactions 

Component transactions enable the learner to 
acquire all of the components which comprise 
a single knowledge frame. The term acquire In 
this context has a range of meanings all the 
way from denote, 8 that Is, remembering or 
recognizing the steps in an activity, or events 
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In a process; to being able to actually perform 
the activity, or interpret a process by 
predicting what will happen in a given 
situation or explaining what is happening in a 
given situation. The level of performance 
required of the learner is a parameter whose 
value is either specified by the instructional 
designer or is determined by the transaction 
configuration or advisor rules of the 
transaction. A component transaction can 
apply to a frame at the instance, class, or 
superclass level. In the later case the 
components being acquired are generalized 
components which can apply in a variety of 
specific cases. 

There are three classes of component 
transactions corresponding to the three types 
of knowledge frames: identify for entity 
frames, execute for activity frames, and 
interpret for process frames. In the following 
paragraphs the performance, knowledge 
required, and a brief description of the 
Interactions supported are described. 

Identify 

Knowledge required. 

An identify transaction requires either an 
Instance or class entity frame. The knowledge 
base Includes a structural representation 
identifying the clustering of parts and 
subparts. The resource data base includes a 
physical representation on which the name, 
location, and function of each part can be 
identified and/or a functional representation 
on which the name, location and function of 
each component can be identified. 
Performance. 

An identify transaction enables the learner to 
acquire the names, functions, properties, and 
relative location of all the parts which 
comprise an entity. The learner knows what it 
is. Learning the names, location, and function 
of the parts of a entity is a prerequisite to 
learning how an entity works, or how to 
operate an entity. Students are shown a 
physical representation of the entity and asked 
to Identify individual parts, their function, and 
their immediate connections. Students are 
shown functional diagrams representing the 
conceptual structure of the entity and asked to 
identify individual components, their 
function, and their immediate connections. 
Students are shown both the physical and 
functional representations and asked to 
demonstrate the correspondence between these 
two representations. 



Interactions 

The transaction must both present all or a 
subset of the names to the learner and enable 
the learner to practice locating the parts and 
identifying the part name and function. 
The transaction must present the functional 
names to the learner, must pair the functional 
names with their physical referents. The 
transaction must enable the learner to practice 
identifying functional symbols given referents; 
referents given the functional symbols; names 
of functional symbol given its graphic 
representation; and reproducing the symbols. 

Execute 

Knowledge Required. 

An execute transaction requires either an 
instance or class activity frame. All activities 
require one or more associated entities which 
are the object of the action or the tools by 
which the action is executed. The knowledge 
base includes a structural representation 
identifying the steps and substeps Involved in 
the activity including the kind of each step 
(activity, action, condition, or loop), the 
components of each step (action, object, tool), 
the initial state of the entities involved, and 
the consequent state of the entities involved. 
The resource data base includes a physical 
and/or functional simulation of the entities 
Involved In the activity such that the actual or 
simulated actions performed by the learner 
result in the illustration of the consequent 
state of the entitles involved. 
Performance: 

An execute transaction enables the learner to 
acquire the steps of an activity. The learner 
knows how and is able to do the activity. 
Execute performance can be at either the denote 
or perform level. At the denote level the 
learner is able to list the steps involved in a 
given activity or shown the activity identify 
the steps and their sequence. At the perform 
level the learner is shown a physical or 
functional simulation of the entities involved 
in the activity and asked to carry out the steps 
necessary to manipulate these entities. 

Execute performance differs depending on the 
type of entities associated with the activity. 
Execute performances associated with object or 
device entities include: device operation 
described by verbs such as run, work, 
manipulate, drive, steer, pilot, play, perform 
on, use, guide; Device assembly or disassembly 
described by verbs such as put together, take 
apart, fix, repair, collect; device adjustment 
described by verbs such as adjust, test, 
calibrate, measure; device trouble shooting 
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described by verbs such as identify 
malfunction, detect faults. The device 
performance often involves one or more tool 
entities which must be used to manipulate an 
application device. 

Execute performances associated with person 
or creature entities include: interact with 
described by verbs such as communicate, 
direct, supervise, control; perform 
characterized by participation in some event. 

Execute performances associated with symbol 
entities include: edit characterized by detecting 
and correcting errors in a symbolic 
communication; compute characterized by 
using arithmetic or mathematics to 
manipulate symbols or solve problems; report 
characterized by preparing a form of 
standardized communication; and compose 
characterized by creating a unique 
communication. 

Interactions. 

The transaction must be enable to present the 
steps of the activity to the learner in a way that 
*he learner can acquire the steps involved, and 
allow the learner to demonstrate their ability 
to identify these steps and their sequence 
and/or their ability to execute these steps in a 
real or simulated situation. 

Interpret 

Knowledge Required. 

An interpret transaction requires either an 
instance or class process frame. All processes 
require one or more associated entities which 
are the agents which cause some 
transformation to occur or are the recipients of 
the transformation. The knowledge base 
includes a structural representation 
identifying the phases, events, and subevents 
involved In the process. Each event includes 
the kind of event (event, episode, condition, or 
loop), the condition (state) of each of the 
involved entities prior to the transformation. 
The relationships which describe the 
transformation. The consequent state of each 
of the involved entities following the 
transformation. The resource data base 
includes a physical and/or functional 
simulation of the entities involved in the 
process such that the transformations 
constituting the events of the process are 
dynamically illustrated for the learner. The 
simulation must have the ability to respond to 
a change in conditions such that the 
consequent change in transformation for a 
given event or series of events Is illustrated. 
The simulation must have the capability for 
the learner to manipulate the conditions of the 



simulation in order to observe and/or predict 
changes in the consequence resulting from this 
change in conditions. 

Performance. 

Interpret performance enables the learner to 
acquire and understand the events in a process. 
The learner knows why it works and can 
explain the events which lead to a given 
consequence or can predict the consequence 
from given conditions. Interpret performance 
can be at either the denote or perform level. At 
the denote level the learner Is able to list the 
events involved In a given process or. shown 
the process, identify the events and their 
sequence. At the perform level the learner is 
shown a physical or functional simulation of 
the entities involved in the process and asked 
predict what will happen under different 
normal and/or faulted conditions and/or to 
identify the conditions upon which a given 
occurrence of an event in the process was 
predicated. Prediction includes the ability to 
recognize when a given process is faulted or not 
functioning as expected. 

Interactions. 

The transaction must be able to present the 
phases and events of the process to the learner 
in a way that the learner can acquire and 
understand the events involved. The 
interactions should allow the learner to engage 
in "what if' exploration of the process by 
adjusting conditions and observing the 
consequences. The transaction must also 
allow the learner to demonstrate their ability 
to identify these events and their sequence 
and/or their ability to predict subsequent 
events from a given set of conditions or to 
identify the prerequisite conditions that 
resulted In a given consequence. 

Abstraction transactions. 

Abstraction transactions enable the learner to 
acquire skills that require the content from a 
class frame and two or more instance frames in 
an abstraction hierarchy. Abstraction 
transactions prdmote the ability to use 9 a skill 
acquired for one set of instances or classes with 
a previously unencountered instance or class. 
Abstraction transactions enable the student to 
generalize their knowledge by acquiring an 
abstraction model knowledge and skills about 
the general case of an entity, activity, or 
process. 

Different types of abstraction transactions can 
be discriminated on the basis of the 
performance required and the different 
combinations of frames from an abstraction 
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hierarchy Involved In the transaction. We 
have identified at least five classes of 
abstraction transactions: Judge, classify, 
decide, generalize, and transfer. We will 
describe only the decide transaction In this 
paper. 

Decide 

Knowledge Required. 

A decide transaction requires a superclass 
entity or activity frame with two or more 
subordinate class frames each of which have 
two or more instance frames. Entitles and 
their parts are characterized by properties 
which can assume a value from a set of legal 
values for that property. The knowledge base 
must Identify the properties associated with 
each entity or its parts and the legal values 
that these properties can assume. Classes of 
entities must Include the properties and values 
which determine class membership. Instances 
of activities must indicate the properties and 
values that are associated with the entities 
prior to the activity and following the activity. 
Performance. 

A decide transaction enables the student to 
know when to select one alternative entity or 
activity from mother. For an entity this 
selection is determined by recognizing and 
selecting those instances which have a certain 
value on the selection properties or set of 
properties. For an activity this selection 
determines an action which is appropriate for 
a set of conditions, that is the value of 
appropriate properties prior to the activity. Or 
by selecting an action which is appropriate to 
achieve a certain consequence, that Is the 
values that certain properties will assume 
following the activity. 

Interaction*. 

The transaction must be able to present the 
entities and their parts, whether for selection 
of the entity or as a condition or consequence 
of an activity, In such a way that the value of 
the properties involved can be illustrated or 
identified from the representation. The 
transaction must allow the learner to 
demonstrate their ability to identify an entity 
or part which has a certain values on a 
particular property or set of properties and to 
indicate the corresponding entity to be selected 
or activity to be performed. 
Association transactions 

Association transactions enable the learner to 
acquire skills that require several different 
associated frames. Association transactions 
promote the ability to integrate information 



from two or more knowledge jrames into a 
coordinated set of knowledge and skill. 
Association transactions enable the learner to 
use a nental model already acquired to build a 
modified or new mental model. Association 
transactions enable the learner to acquire 
alternative ways to accomplish a given goal. 
Association transactions enable the learner to 
Invent new entities or activities or to discover 
new processes. 

Different types of association transactions can 
be discriminated on the basis of the 
performance required and the different 
combinations of frames from a set of 
associated frames involved In the transaction. 
We have identified at least five classes of 
association transactions: propagate, 
analogize, substitute, design, and discover. 10 

Enterprises 

A complex interrelated human activity 
requiring a combination of knowledge and 
skills Is called an enterprise. A primary goal of 
most instruction is to enable a learner to 
acquire the knowledge and skill required to 
engage In some enterprise. 

We are currently engaged in a project to design 
and build transaction shells to train aircraft 
maintenance personnel. A consultant to the 
project has identified a number of enterprises 
that are required for aircraft maintenance 
(Halff, 1990). These include the following: 

"Operation. In most maintenance contexts the 
malntalner must be able to operate, to some 
degree, the equipment being maintained. 
Operational skills are used to verify the status 
of the equipment, to prepare the equipment for 
maintenance, and to interpret reports from 
operators. 

"Calibration and adjustment. Many devices 
must be configured for particular operating 
environments, calibrated, and adjusted on 
occasion. Maintenance personnel are 
routinely called on to effect such adjustments. 
These adjustments are often a part of 
preventive maintenance, and they often 
constitute repairs. 

"Testing. Equipment testing is a critical part of 
maintenance. Malntainers must be able to test 
an equipment's operational status. They must 
also be able to conduct particular diagnostic 
tests during the course of troubleshooting. 
These tests often require the use of general 
purpose and specialized test equipment, and 
this test equipment must itself be properly 
calibrated and operated. 
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"Access and disassembly. In the course of 
repair, testing, and calibration, maintainers 
must gain access to particular components for 
observation and manipulation. The 
procedures ussd to gain access can be 
straightforward in some cases. In others, 
special procedures are required to ensure that 
gaining access to one part of the equipment will 
not damage other parts. These procedures are 
normally specified by the manufacturer of the 
device. 

"Repair. By repair, I mean the operations 
needed to return a device to operability once a 
fault has been isolated. Repairs therefore 
include replacement of faulted components, 
cleaning, adjustment, patching, and a host of 
other operations. 

"Troubleshooting. Perhaps the most 
challenging maintenance operation, from a 
training viewpoint is that of troubleshooting. 
Troubleshooting Is the process of identifying 
the physical cause (fault) of an existing or 
potential malfunction of the equipment's 
operational capabilities. For the most part, 
troubleshooting takes place after a 
malfunction occurs, but troubleshooting also 
comes into play when a test -- say, during 
preventive maintenance reveals a potential 
fault." 

In order to engage in any of these maintenance 
enterprises the learner must first acquire a 
mental model of how a particular device 
appears and operates. An adequate mental 
model of device functioning involves the 
following capabilities: 

Device structure, "...shown Images of the 
physical equipment ...[learners are able tol 
identify individual components, their 
function, and their immediate connections." 

Device function. "... learners are (able) to 
discriminate among component states on the 
basis of some physical depiction of those 
states." 

Device configuration, "...shown some of the 
inputs to an element of the device Deamers are 
able to show] ... how its other inputs must be set 
in order to achieve a desired function or state." 

Fault recognition, "...shown the actual outputs 
and inputs to an element [learners are able 
to)... determine whether or not the element is 
faulted." 



Prediction, "...given information about all 
inputs to a component or subsystem [learners 
are able tol... predict the state of the component 
or subsyste n, its outputs under normal 
operating conditions, and its outputs in each 
possible fault mode." 

Transaction Families 

Acquiring a mental model that enables a 
learner to perform some complex enterprise 
involves interacting with a number of different 
instructional transactions. Most instruction 
is accomplished by a group of transactions all 
working together to convey the knowledge 
required by the enterprise. The group of 
transactions required to teach a given 
enterprise is called a transaction family. A 
transaction family works via an enterprise 
transaction. An enterprise transaction is a 
higher level transaction which accomplishes 
two purposes: first, it functions as a 
transaction manager, providing the overall 
direction of the execution of the individual 
transaction instances in the transaction 
family. Second, it provides for an integration 
of the learning facilitated by the individual 
transactions In the transaction family. 

In a previous section we identified several 
classes of transactions. Each of these classes 
can have a number of specific instances whfc h 
share all of the characteristics of the class but 
which is tailored for a particular type of subject 
matter. For example, we identified an identify 
class of transaction; in a maintenance 
training environment a transaction designed 
to teach device structure is an instance of an 
identify class transaction, but would have 
characteristics peculiar to teaching about 
devices. We identified an interpret class of 
transaction; in a maintenance training 
environment a transaction designed to teach 
device functioning Is an instance of this class, 
but one that has characteristics peculiar to 
device functioning. Other instances of 
interpret class transactions include a 
transaction for device configuration, fault 
recognition, and prediction. Each of these 
instances share the characteristics of an 
interpret transaction class but have additional 
characteristics peculiar to their particular 
domain. 

Figure 2 Illustrates a transaction family for 
teaching the learner a model of a device. This 
transaction family consists of an enterprise 
transaction that Is peculiar to equipment 
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models, and which manages 5 other 
transactions: a device structure transaction of 
the class identify; and device function, device 
configuration, fault recognition, and 
prediction transactions of the class interpret, 

Figure 3 Illustrates a transaction family for 
teaching the learner the enterprise of 
equipment troubleshooting. This Is a much 
more complex transaction family which 
Involves the nesting of other transaction 
families within its scope. The troubleshooting 
enterprise transaction determines If the 
learner already has an adequate equipment 
model, If not, it engages the equipment model 
transaction family to assist the learner in 
acquiring this model. The troubleshooting 
enterprise transaction determines if 
disassembly is necessary for the 
troubleshooting procedure being taught, if yes, 
it determines if the learner has the necessary 
disassembly skills, if not, it engages the 
disassembly family of transactions in order 
for the student to acquire the necessary 
disassembly skills 11 . The troubleshooting 
enterprise transaction then engages a logical 
fault isolation procedure transaction which is 
an instances of the execute class. If the learner 
must learn to select from alternative fault 
isolation procedures it also engages a fault 
isolation procedure selection transaction 
which is an instance of the decision class. If 
the troubleshooting requires operating the 
equipment the equipment operation 
transaction family may be engaged; if repair is 
involved the equipment repair transaction 



family is engaged. The instruction continues 
in this manner with the troubleshooting 
enterprise transaction engaging other 
enterprise transactions that in turn engage 
their Individual transactions as necessary 
until the learner has demonstrated his 
acquisition of the necessary knowledge and 
skill. 

Summary 

Instructional transactions are instructional 
algorithms, patterns of learner interactions 
which have been designed to enable the learner 
to acquire a certain kind of knowledge or skill. 
A transaction shell is computer code that 
when provided to a student via an appropriate 
delivery system, causes a transaction or set of 
transactions to occur. 

A transaction shell consists of four major 
components: Interactions and an interaction 
manager, instructional parameters, a 
knowledge base and a resource data base. It 
also requires three authoring systems: a 
transaction configuration system, a knowledge 
acquisition system, and resource editors. 
Transaction shells capture instructional 
design intelligence via methods for 
Interactions, knowledge base syntax, rules in 
the knowledge acquisition system, parameters, 
and instructional design rules in the 
transaction configuration system. 

All instructional transactions must include 
the following functions: knowledge selection, 
knowledge sequence, instructional 
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management, and instructional enactment 
Each of these functions is accomplished via 
methods and parameters which constrain 
these methods for a particular transaction 
class. 



A transaction class is a 
transaction shells which 
interaction requirements 
knowledge representation 



set of similar 
have similar 
and similar 
requirements. 

Based orTthe type of knowledge taught three 
primary classes of transactions have been 
identified: component transactions, 
abstraction transactions, and association 
transactions. Each of these primary classes 
include several subclasses. In this paper the 
knowledge, performance, and interactions 



required for identify, execute, interpret, and 
decide classes of transactions are briefly 
described. 

A transaction family is all of the transactions 
necessary to enable a learner to acquire all of 
the knowledge and skill required to engage in a 
particular complex human activity called an 
enterprise. An enterprise transaction is a 
higher order transaction that manages a 
family of transactions which acting together 
enable the student to acquire the knowledge 
and skill required by the enterprise. 
Enterprises for maintenance training and 
some representative transaction families were 
briefly described. 
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Abstract 

This paper reviews the Courseware 
Development Laboratory (CDL) at Utah State 
University, and Second Generation 
Instructional Design Theory (ID2) as developed 
by Merrill, Li, and Jones. The first practical 
application of ID2 is being implemented at the 
lab. Training materials are being developed for 
a corporate client by faculty and graduate 
students. The paper briefly describes the CDL 
concept, gives a short overview of ID2, and 
reports on the implementation project and its 
results. 

In the Department of Instructional Technology 
at Utah State University, Instructional 
designers, professors, graduate students and 
media production specialists combine to make 
possible a unique courseware development 
laboratory. This laboratory enables some of 
the most talented people available to produce 
commercial quality software which explores 
the latest innovative techniques in educational 
software design. 

Background 

The Department of Instructional Technology at 
USU offers a series of graduate level classes in 
instructional design and a parallel series of 
laboratory experiences in the design and 
development of computer based instructional 
materials. Beyond the introductory courses 
these are laboratory based classes in which 
students and faculty design and produce 
prototypes of instructional products. 

The Courseware Development Laboratory (CDL) 
The Courseware Development Laboratory 
forms a meta structure separate from, yet 
integrated with the academic courses. 
Emphasis is placed on the development of 
instructional products based on Instructional 
design and on going research concepts. The 
goal is practical application of the latest 
Instructional design theory, giving graduate 
students hands-on experience with a real 
product for a real client. Every attempt is made 
to simulate a corporate design team experience. 

Students selected to participate in the CDL 



program must have had or anticipate taking 
the relevant design, production and evaluation 
classes at least one quarter prior to the use of 
this Information in the CDL experience. A long 
term commitment, a minimum of nine 
months, (three successive quarters) is expected 
of each student. Students are assigned to teams 
consisting of four to five students. Each team 
has a team leader who is a PHD student and 
who is responsible for directing the work of the 
team. After being assigned to a specific team, 
which will work on a specific product for a 
specific client, the work phase progresses as 
follows: 

1st phase - Analysis and Design. (3 months) 
2nd phase - Product Development and 

Documentation. (3 months) 
3rd phase - Implementation and Evaluation. (3 

months) 

The enc result of the nine month commitment 
is a finished product which is delivered to a 
client. (Courseware Development Laboratory, 
1990). 

Second Generation Instructional Design 
Theory (ID2) 

For many years traditional instructional 
design has been very useful for the development 
of quality instructional products. Most 
traditional design theories were developed 
before the so called "computer revolution", and 
as a result several limitations have been 
identified. These include: 

* Development of CBI is very labor intensive, 
often requiring 200 or more hours of 
development time to produce one hour of 
computer based Instruction. 

* Prescriptions for course organization 
strategies using the potential of computer 
based Instruction are superficial or often 
nonexistent. 

* There is a tendency to focus on, and thus 
teach, components of content rather than 
Integrated wholes. 

* There are limited or no prescriptions for 
knowledge acquisition. 

* It Is difficult to accommodate new knowledge 
about instruction, or the learner, as it 
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becomes available. 

* Each phase of Instructional development is 
performed essentially independent of other 
phases and there are no means provided for 
integration or sharing of data. 

* The development process Is very inefficient in 
that the instructional designer must often 
build every presentation from scratch. 
(Merrill, Li, & Jones, 1990) 

Second Generation Instructional Design 
addresses each of the limitations listed above 
with the net result being much greater 
efficiency In the Instructional design process. 
The principles of ID2 are especially applicable 
to computer-based instructional design and 
development. 

n>2 Instructional Development Process 

Figure 1 illustrates the instructional 
development process under ID2. 

There are three major components in the ID2 
Instructional development process, 
represented in figure 1 as square cornered 
boxes: 1. KAAS (Knowledge Acquisition and 
Analysis System). 2. SAS (Strategy Analysis 
System), and 3. Transaction Authoring. 

The three ovals identify three subcomponents 
that contribute to the instructional 
development precess: 1. Domain Knowledge 
Base (elaborated frame network), 2. Course 
Structure (transaction network), and 3. the 
actual finished course (transactions). 

The three remaining curved boxes identify the 
"human factor" interaction with the system. 

How it Works 

The subject matter experts (SME'sj and 
designers work together to identify and record 
all the "dentin knowledge" that is required for 
any given course. This knowledge is entered 
into the KAAS computer program. KAAS 
contains knowledge acquisition techniques 
which assist and guide the entry process sc that 
a Domain Knowledge Base is created. This 
knowledge base is configured and coded In such 
a way that it can be accessed by ;he Transaction 
Authoring system. 

Designers and production personnel work with 
ihe Transaction Authoring computer program 
to create the actual course presentations 
(transactions). The authoring process has 
access to not only the Domain Knowledge Base, 
but also the Strategy Analysis System (SAS) 
and the transaction network from the Course 
Structure component of the system. 



Two important considerations are included in 
the development process at this point: 
1. SAS, a computer program that the designer 
can Interact with, accepts and stores such 
information as; training goals, student 
information, environment information, 
instructional design strategies, mini expert 
systems, etc. The designer can select from a 
variety of presentation modes those 
instructional techniques that best fit the 
particular subject being considered. 2. The 
second point relates to the transaction 
network. One of the powerful tools available in 
the ID2 concept is the notion of transaction 
shells. These are shells or containers that have 
been previously designed and configured by 
instructional technologists/programmers. 
They are available in a variety of formats to 
receive the specific course content that is 
necessary for a particular course of 
instruction. This, of course, infers that there 
are many transaction shells organized into 
groups or "families" that pertain to specific 
subject requirements. The selection of this pre- 
existing shell enables the designer to by-pass 
the labor intensive "creating from scratch" 
function of traditional instructional design, 
thus realizing a considerable savings in 
development time. These transaction shells 
exist in two modes, authoring and delivery. 
(Merrill, U.uid Jones 1990) 

CDL and Second Generation Instructional 
Design 

The focus of this paper is the application of ID2 
in the Courseware Development Laboratory. 

A corporate client contracted with Utah State 
University for the development and production 
of some internal training materials using the 
1D2 theory. The project was to focus on two 
specific Instructional packages that had been 
developed by the clients internal training 
personnel. The client has an extensive 
corporate education organization which 
employs numerous instructional designers, 
subject matter experts, and trainers. The 
materials under consideration were in use 
within the corporate education system. One of 
the significant goals of the project was to study 
the ISD process under ID2 theory. 

Goals of the Project 

It was decided to integrate the corporate project 
with the Courseware Dev lopment Lab at Utah 
State University. The following goals or 
outcomes have been identified: 
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First, this will be the first practical application 
of the second generation instructional design 
theory (ID2) in developing a real piece of 
instruction. Observations will be made in 
order to learn how ID2 might change the 
traditional ISD process. 

Second, we will collect data on how beginner 
(novice) Instructional designers Interact with 
the KAAS tool. Also we will test the 
completeness of the knowledge representation 
model underlying KAAS. Both of these 
evaluations will contribute to the novice 
version of the KAAS tool. 

Third, we will focus on the development of up to 
five hours of instruction that can be directly 
used to training by the corporate client. We 
will also evaluate the instruction and the 
efficiency of developing such Instruction, both 
which will be an extra benefit to the corporate 
client. 

Fourth, the project will give the selected 
graduate students practical experience in 
applying ID2 theory. Students will also gain 
first hand experience in a "corporate design 
atmosphere". 

Fifth. this Joint Utah State 
University/Corporate project will be examined 
as a model for corporate partnership with 
higher education. 

Expectations and Results 

In order to meet the first goal, developing a real 
piece of Instruction using ID2 theory, it was 
necessary to obtain samples of the corporate 
training materials including written 
documentation. After a period of initial 
orientation and up front analysis with this 
documentation, three specialists from the 
education division of the corporation made an 
on-site visit t .ie university and spent three 
days with the CDL team. Work sessions were 
scheduled between the corporate team and the 
graduate students. The purpose of these 
meetings was to review the knowledge base 
behind the training materials under study. The 
corporate team acted as Subject Matter Experts 
(SME's) and the CDL team members 
interviewed them concerning the training 
content and all aspects of the organization of 
the training materials. The three day visit 
provided a number of separate opportunities 
for the groups to interact, record data, ponder 
and analyze what had been discussed, 
formulate new questions, and Interact again. 
This process produced excellent results and the 
CDL development team came away with a much 



clearer understanding of the training 
materials. 

The second goal was directed toward the KAAS 
tool. The students in the CDL team were 
trained to the use of the KAAS tool. Following 
the analysis of the corporate training 
materials the students will be required to enter 
the data Into KASS. Observations will be made 
as the novice student designers work with 
KAAS and data will be collected about their 
experiences. Also, careful evaluation will be 
conducted with the knowledge representation 
model underlying KAAS to see how it handles 
the specific domain knowledge of the corporate 
training materials. 

( Note - the results of the interaction between 
the novice designers and KAAS, and the 
examination of the knowledge representation 
model will be completed and documented later 
to the study.) 

The third goal, the development of up to five 
hours of Instruction for the corporate client, 
will be Implemented during the second phase of 
the project, the Product Development and 
Documentation phase. 

The intent of this phase of the program is for 
the graduate students, to consultation with the 
ID2 theorists, to produce the specifications for 
the required transaction shells and have these 
specifications programmed as functional 
shells by the software engineers who are 
assigned to the project. Following the actual 
production of the working shells the CDL team 
will then undertake a formative evaluation of 
the first prototype and make revisions as 
necessary. This goal also requires an 
evaluation of the instruction and the efficiency 
of developing such instruction. This will take 
place during the third phase of the project, the 
Product Implementation and Evaluation 
phase. 

(Results and documentation of this phase will 
be available following the completion of the 
project to the summer of 1991.) 

The fourth goal of the project, giving the 
graduate students practical experience in 
applying ID2 theory, will be significant to their 
education as Instructional technologists and 
will better prepare them to move Into education 
or the corporate world as well trained designers 
of Instruction. 

The fifth outcome of the project, the model for 
corporate partnership with higher education, 
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is especially significant to the continuation of 
the development of new Instructional design 
theory and the Implementation of new and 
better Instructional design techniques. 

Summary and Conclusion 

The theory and underlying principles of ID2 
have Important application for any 
Instructional design process, especially 
Instruction that Is computer-based. The 
development of computer-based transaction 
shells and a knowledge representation system 
can be a very complex task, but once developed, 
will be tremendous tools for the Instructional 
design and development process. Specifically, 
ID2 provides the Instructional designer with 
powerful tools for analyzing the knowledge 
that needs to be taught, the strategies and 
transactions that would most effectively teach 
a given subject and audience, and a systematic 
and easier way to author a course. (LI and 
Merrill, 1990) 

The Courseware Development Laboratory at 
Utah State University provides a unique 
opportunity for advanced graduate students to 
interact with some of the leading Instructional 
design theorerlsts as they expand the 
boundaries of Second Generation Instructional 
Design. The ultimate reward comes as the CDL 
team has the opportunity to effect one of the 
first practical implementations of ID2 theory 
for a corporate client in the real world of 
design, development, and utilization. 
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Abstract 

This paper presents an alternative approach 
for the design, development and 
implementation of Computer Based Training 
(CBT) applications, based on concepts 
developed in the production of courseware to 
teach the principles of CBT (meta-CBT). During 
the design phase, it became evident that the 
traditional approach to information 
presentation based directly on instructional 
objectives was inadequate, giving rise to the 
notion of transformational Instructional 
Systems Development, in which the 
courseware structure is based on discrete 
performance events rather than content or 
task hierarchies. A presentation strategy was 
implemented in which the learner was an 
integral component of the instructional 
interactions, playing an active role in the 
selection and manipulation of content. This 
strategy formed the framework for a new 
presentation model, the Learner Integrated 
Training Environment (LITE), which provides 
courseware developers with alternative options 
for structuring and delivering courseware. 
Given the demand for courseware 
Implementations using this model, the LITE 
demonstrates a new paradigm for courseware 
presentation, especially in the work 
environment, where effective performance is a 
vital factor for success. 

Towards Alternatives iter CBT 

Much of past, present (and perhaps future) 
education has been based on the instructor's 
perception of content and discipline. The 
traditional lecture and tutorial have provided a 
framework and environment for the teacher to 
emphasise their particular understanding in 
the context of the unit of instruction being 
presented. This approach has also frequently 
been reflected in many of the commercial 
computer-based educational applications, with 
the underlying structure resulting from the 
production techniques of Instructional 
Systems Development (ISD) methodologies 
(Reigeluth, 1983; Dick & Carey, 1985; Gagn*. 
Brlggs & Wager, 1988; Romiszowski, 1988). 

With frequent debate as to the effectiveness of 
CBT and the current emphasis on productivity 
and performance, it is appropriate to question 
the traditional models of computer-based 



training, and the instructional design systems 
which created them. A typical norm for 
courseware is one in which students are guided 
through a content domain using instructor- 
initiated prompts and cues. This content- 
question-feedback model is illustrated in 
Figure 1 (based on Alessi & Trollip, 1991). 
While this approach has become accepted 
practice, it is evident from the development of 
many different courseware applications that 
the resulting interactions are often 
unsatisfactory as learning events, and the 
cause of the frequent critical reports of CBT 
courseware (Sims, 1986a, 1986b, 1991a). 

In contrast, recent emphasis has been placed 
on the learner and their performance in the 
work environment (Sims, 1991a). This is 
consistent with the directions of Performance 
Support Systems (Gery, 1987) and Embedded 
CBT (Bentley, 1990), which focus more on the 
learner's effective performance in the work 
environment, rather than their demonstration 
of content mastery in off-the-Job training 
sessions. 

Using the Australian work environment as an 
illustration, there have been significant 
developments in Job restructuring and multi- 
skilllng, leading to extensive definitions of 
performance levels and competency-based 
training. However, it is apparent from 
materials produced by industry groups that the 
focus is on content knowledge, rather than 
workplace performance. For example, in the 
hospitality industry, one of the defined 
competencies is to identify different types of 
service providers (restaurant, hotel, bar etc), 
although there is no clear context or purpose 
for this knowledge. While comprehensive, it is 
predictable that this framework for Australian 
industry training may generate similar 
problems to that of computer education: over 
emphasis. Solutions such as those proposed by 
Carroll (1984, 1990), in conjunction with those 
presented in the current discussion, are 
relevant to the effective implementation of new 
training and performance standards. 

Extending the argument to courseware 
development specifically leads to the 
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Figure 1 Traditional Tutorial Model 

prediction that flaws In courseware modules 
may be due to an exaggerated emphasis on 
instructional objectives and hierarchical task 
structures. The suggestion Is that a solution is 
to present the content In such a way that it will 
maximise opportunities for learning, 
exploration and discovery In the context of job 
performance. 

To support this idea, Figure 2 illustrates an 
integrated presentation strategy whereby 
students have greater flexibility In selecting 
options for working through instructional 
content (Sims. 1991b). This strategy Includes 
elements of Artificial Intelligence through the 
impleme 

ntation of a microworld and knowledge-base to 
reflect a particular work environment 
(Kearsley, 1987). In essence, the student 
experiences a simulated work environment, 
and learns by problem-solving and decision- 
making through typical work scenarios. While 
similar to the simulation models proposed by 
Alessi & Trollip (1991), the approach suggested 
is one in which the student Is the focus, in the 
same way as the employee is the focus in the 
work environment. 

This integrated model emphasis. - move from 
an Instructor-led approach to ue which is 
student-centred and student-controlled: a 
paradigm shift from the traditional 
behavioural models to cognitive operations. 
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Figure 2 Integreated Tutorial Model 



Although traditional approaches to courseware 
development have identified specific 
presentation formats for different training 
requirements (tutorial, drill, simulation etc), 
this discussion argues that content material 
may be presented more effectively using 
different and new techniques. More 
importantly, it questions the requirement for 
content presentations to require specific prior 
knowledge for interpretation and 
manipulation, as the students will often be 
mature, literate and capable adults. While 
students do require specific guidance, there is 
Increasing evidence and thought that learning 
will be more effective when knowledge is 
gained through individual pursuit and 
endeavour (Schank & Farrell. 1988). 

The Meta-CBT Project 

An opportunity to apply these concepts arose 
with the need to develop a course about 
Computer-Based Training (CBT) using 
courseware as the delivery medium (meta-CBT). 
The demand for the course came partly from 
training departments who needed to convince 
management of the value, features and 
processes of CBT. and partly as a result of the 
various tertiary courses which are now 
teaching CBT. In both cases, few suitable 
materials were available to present the 
fundamental concepts. 
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In fact, the major techniques for presenting 
CBT as a concept have tended to focus on 
authoring tools and exemplary courseware 
rather than the significant factors of the CBT 
development process. In brief, the major 
options for presenting CBT have been: 

Demonstrating Authoring Tools 

While of Interest, an authoring tool is not a 
training resource, and the demonstration 
software Is likely to overwhelm all but the 
most experienced authors as it will go beyond 
the scope of the project and the ability of the 
development team. 

emonstratlng Samples of CBT Courseware 

completed products can be impressive, but they 
are all too frequently irrelevant to the current 
corporate need. It is vital to impress upon any 
decision makers the benefits and 
commitments CBT will bring to the 
organisation. 

Training Sessions on CBT 

While practical, there is the risk that material 
presented will be too general and not enable the 
major Issues and factors associated with CBT 
development to be explored. 

One of the items not listed is CBT courseware, 
and it appears there has been little application 
of CBT as the medium of explanation for CBT! 
in an attempt to provide a viable and 
contextual environment for the potential 
student (senior executives, trainers, students of 
CBT), the current technology was assessed to 
determine how it could be ' •? used. The most 
obvious features considered w .re: 

Multi-media and Graphics 

This provides a variety of presentation options 
such as animation, video images and audio. 
One major advantage of these options was that 
they could minimise the amount of text 
displayed to the student during instruction as 
well as enhance the graphic display. 

Organisational metaphors 

With significant emphasis on screen design, 
there has been little use of the technique which 
presents courseware using a context with which 
the student is familiar, such as an office, 
workshop or personal work area. The perceived 
benefit of the metaphor was that it would 
enhance courseware realism or fidelity. 

Interactive environments 

This has always been the feature of CBT, and 
yet the medium has most frequently been 
criticised for electronic page -turning. The 



principle adopted for the project was that 
students would be involved in some activity or 
decision-making process throughout the 
courseware. 

However, apart from the technologies 
available, a major consideration of the 
development process concerned relevance, in 
terms of the factors of CBT which should be 
understood by either management or 
prospective users. This technique is used by 
information retrieval specialists, who also 
work with the additional dimension of 
precision (Lowry, 1990). In transferring this 
approach to courseware development, it was 
vital that the material presented to students 
was both precise and relevant. 

In assessing the relevant factors for CBT, the 
traditional approach has been to focus on the 
features: colour, Interaction, response Judging 
etc (Sims, 1990). But this is only part of the 
finished product. What has to be understood, 
especially by organisations and students, is the 
actual process of courseware production, from 
Initial concept to final installed product. 
Therefore, it was necessary to re-assess CBT to 
determine how and what should be included, 
based on the understanding that CBT, as a 
topic, is not Just courseware but the complete 
development process. After careful 
consideration, It was decided that rather than 
implement a tutorial model to cover the basic 
topics of CBT, the courseware would be 
developed to integrate students into the CBT 
development process. The major constructs of 
the courseware were as follows: 

Integration 

The courseware would not be tutorial (didactic) 
in nature, but designed so that the student 
would be integrated into the CBT development 
process and environment. 

Strategy 

Instead of telling the student about CBT, the 
strategy adopted was based on a discovery 
approach, in which the student became 
involved in the actual design, development and 
use of a module of courseware. 

DataBase 

The courseware was designed to contain a data- 
base of screen displays and Interactions 
relevant to an organisational training course 
in product training. Throughout the 
courseware, the student would "assist" with the 
development of these courseware objects. 
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Figure 3a Learner Integration 
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Figure 3c Finished Product 

Organlaational Metaphor 

The student was "tatroduced" to members of the 
courseware development team, and assisted 
them In their various tasks (such as 
Instructional design; specifying objectives, 
selecting screen designs, designing the learner- 
computer interface). Figure 3(a) illustrates a 
display in which the student is about to make a 
decision on screen design. 

Control 

If the student moved out of sequence (that Is, 
required additional information to complete a 
task), it would be suggested that they discuss the 
project with another team member's work; in 
other words, the team member would indicate 
they could not continue with the production of 
courseware without additional details. 
Interaction 

The student was shown what a team member 
was doing, and asked to make a selection or 
decision on their preferred option (objective, 
screen design, Interaction, animation, multi- 
media display etc). Figure 3(b) illustrates 
possible feedback resulting from such a choice. 



Figure 4 Sample of Control Logic 
Creativity 

After making a selection, the courseware object 
(such as a specific screen) would be identified 
for Inclusion In the finished product, and 
integrated into the sample course. 

Involvement 

On completion of the meta-CBT courseware, the 
student had contributed to all facets of the 
courseware development process and In doing 
so, created a lesson which could be viewed. 
Figure 3(c) illustrates a screen from the 
completed product. 

The principle of this meta-CBT was to integrate 
the student into the actual process in order that 
they may experience first hand the nature of 
CBT design, development and production. In 
addition, the course focused on aspects such as 
cost Justification and the resolution of training 
problems in terms of the application of CBT 
rather than other training resources. 

One important factor Is that the meta-CBT 
implementation does not imply that the design 
and structure of the interactions is free from 
standards and instructional integrity; 
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however, it is necessary to differentiate the role 
of the learner and the operation of the 
courseware. One the one hand, the learner must 
have access to courseware which enables them 
to move freely within the content area, and to 
simply work with the interactions presented. 
On the other hand, the courseware must record 
the information presented, depending on 
which items the student has completed, so that 
the interactions are relevant to the student's 
progress through the material. 

In this particular course, the courseware 
allowed the student to interact with the CBT 
production team at any of the 5 major 
production stages (analysis, design, 
development, Implementation and evaluation). 
However, it was also integral to the structure 
that the student could not view the completed 
module until each of the stages had been 
considered and in the correct sequence. To 
achieve this, a high-degree of logic control (i.e. 
programming) was necessary to ensure that 
certain conditions were met, and to ensure the 
processes of CBT development were 
emphasised. 

For example, if the student were to work with 
development tasks before completing the 
design activities, the Importance of the design 
function would be lost. The courseware 
therefore had to maintain a record of the 
courseware development tasks completed by 
the student. The basic logic for this particular 
case is shown in Figure 4. 

While this is only illustrative, and does not 
cover all of the many options available to the 
student, it does emphasise that to develop a 
more effective courseware environment 
requires considerable expertise in courseware 
development and authoring. In this case, the 
meta-CBT courseware covered new concepts in 
the presentation of instructional interactions, 
but to achieve this required the use of complex 
presentation logic. 

Transformational I8D 

One outcome from the work with the meta-CBT 
courseware was the concept of 
transformational Instructional Systems 
Development (ISD), which extends the 
traditional task analysis and statement of 
objectives to a process whereby the content is 
expressed as a work event, and objectives 
linked to 

that event addressed by the courseware. This 
varies from the more traditional and accepted 
approaches to ISD, which break down the 
content material into manageable units or 



tasks, and objectives (both enabling and 
terminal) are generated to provide a measure of 
performance on those tasks. 

The mistake made by many courseware 
developers Is to use tl ose objectives as the focus 
and structure for the design process, wlh the 
result that the instructional sequences 
developed reflect the content its associated 
objectives rather than the context of the 
content and its relevance to work activities. In 
terms of courseware presentation, there is the 
potential that the material will be too content 
oriented and lack any specific continuity in 
terms of Job performance. Figure 5 illustrates a 
traditional lesson structure with a number of 
objectives, and the way in which those 
objectives may be extracted and integrated into 
an instructional event which is based on a 
specific performance activity. 

The importance of the notion of 
transformational ISD is that additional 
consideration must be made after completing 
the traditional task analysis phases (Sims, 
1991b). The content and objectives will then be 
grouped in terms of Job performance rather 
than content hierarchy, which will provide an 
environment in which the student can work 
with the material as it would be found on-the- 
job. 

The Learner Integrated Training Environment 
(LITE) 

In practical terms, the development of 
alternate courseware strategies reflects a 
paradigm shift (from behavioural to cognitive), 
and illustrates a movement towards focusing 
on the learner rather than the 
instructor/content (Sims, 1991a). As shown in 
the previous discussion, this will result in a 
change in courseware structure and 
appearance. In many ways, this "new" 
courseware will be similar to some of the 
original exemplary models of CAL which were 
geared towards individual analysis and 
problem solving, and which mainframe PIATO 
users may remember. It is possible that the 
approach suggested in this discussion has been 
neglected because it requires additional effort 
and extensive programming skills rather than 
for pedagogical reasons. 
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Figure 5 Transformational ISD 



However, based on the meta-CBT development, 
it is anticipated that this courseware format 
will become the standard, and will focus on 
different instructional parameters, where the 
emphasis is on the activities the learner will be 
required to complete to demonstrate successful 
performance. This will be accomplished by 
placing the learner into the subject or work 
environment where they will be required to act 
upon activities (both planned and random) 
which occur naturally in that environment. 
This Learner Integrated Training Environment 
(LITE) is designed to provide a true fidelity 
simulated environment where the focus of the 
interactions is the student and the outcomes of 
their actions. The basic structure of the LITE is 
illustrated in Figure 6. 

While this is a development and modification 
of the simulation format of CBT, it is also an 
attempt to address the fact that CBT has not 
been successful. The LITE environment is 
suggested as a way to re-think the way CBT (and 
other Instructional resources) are designed and 
implemented. 

Based on these concepts of learner integration, 
one of Australia's banks has adopted the model 
to address a training situation which had been 
treated with traditional CBT. The courseware 
had been designed to train branch supervisors 
in their roles as training coordinators; 
however, it became evident that they were not 
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Figure 6 Learner Integrated Training 
Environment 

managing the training process as required. To 
address the situation, the LITE model was 
Implemented whereby the courseware placed 
the supervisor directly into the work situation 
and allowed them to experience the 
responsibilities and outcomes of their actions. 

In another example of this process, a recent 
courseware development project concerning 
proprietary electronic-mail software was 
developed using the LITE principles. While the 
content was based on the software manual, the 
presentation sequences had little relation t- 
the sequence of the manual. The structure of 
the courseware was based on students reacting 
to communication tasks resulting from work 
situations, and the electronic mall software 
was accessed to solve the problem. One of the 
outcomes of the design of this application was 
the presentation metaphor illustrated in 
Figure 7. 

In addition, the actual purpose of the course 
received considerable scrutiny to determine the 
exact purpose of the courseware. In the case of 
the electronic mall training, one of the major 
objectives was to change attitudes by 
demonstrating the effectiveness of electronic 
communications. Thus the course was titled 
"Have you checked your mall today" (with the 
corporate logo) rather than using the 
traditional "An Introduction to Electronic 
Mail". 
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In summary, the electronic mall courseware 
focused on using the software to achieve 
particular work requests, rather than how to 
use the software, which was treated as 
subsidiary to the communication task to be 
completed. In addition, the material was 
presented in a realistic manner, without 
significant assistance, which is what would 
occur in the real work environment. 
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Figure 7 LITE Example 

This use of a question as a title for courseware 
applications can assist in determining strategy 
and direction. For example, in the banking 
industry, a trainee teller will receive a unit of 
Instruction on the withdrawal slip. 
Traditionally, this will be a series of examples 
describing each of the sections on the slip 
(name, account number, signature etc). 
However, when trainers are questioned as to 
"why" they conduct this training, there Is often 
no answer. (Interestingly, this is one of the 
most difficult aspects of the courseware 
development process: identifying the 
instructional discrepancy which initiated the 
original demand for the courseware). 
Therefore, to Implement a LITE, the course 
could be transformed such that the content was 
expressed in terms of "Will the customer receive 
their money" rather than Teller Training 
Module 75: 

The Withdrawal Slip 

This technique of expressing the content In 
terms of a question is integral to the LITE 
model, as It allows the designer to focus on the 
performance required by the student. In 
addition, this approach will typically result to 
different combinations of content as well as 
emphasising the relevance and purpose of the 
training. 



Future Directions 

The next stage In the application of 
instructional technology Is the successful 
integration of multi-media components. It Is 
likely that future courseware applications will 
see a significant reduction In displayed text, 
replaced by digitised audio, with graphics and 
Interactions incorporating DVI or similar 
technologies. With respect to presentation 
strategies, models such as LITE will continue to 
be enhanced, applying techniques such as those 
used to the popular adventure game. It is 
Important to highlight the observation that 
game players can spend many hours trying to 
solve problems, and to doing so, learn the tasks 
required to solve particular problems within 
the game. There is no reason why this strategy 
should not be applied to courseware 
applications. 

Given these developments In courseware 
production and presentation, the task for 
trainers Is therefore to assess the desired 
outcomes of the training to terms of student 
performance, rather than a focusing on a direct 
demonstration of knowledge. The introduction 
of new instructional strategies for CBT is 
therefore essential, especially with the 
developments in presentation technology and 
the increased demand for effective training and 
performance. The meta-CBT project, the 
resulting concept of transformational ISD and 
the development of the Learner Integrated 
Training Environment model emphasise the 
advantages of using alternative options as a 
means for enabling students to work with new 
concepts and tasks. 
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PRESCRIPTIONS IN READING COMPUTER-ASSISTED 
INSTRUCTION: READING VERSUS WRITING 



Roy B. Clartana, EG&G, Rocky Flats Inc. 



Abstract 

This study was designed to answer specific 
questions regarding the use of extensive 
computer curriculum products in computer 
laboratory settings In the upper elementary 
school. Should a "variety" of curricula be 
provided? And. does a computer laboratory 
setting impact girls and boys differently? 
Fifth- and sixth -grade students (n=115) were 
randomly assigned to either a reading only 
treatment (RO) or a reading and writing 
treatment (RW). The RO group received the 
WICAT Reading Comprehension product twice 
weekly throughout the study. The RW group 
received the same reading product once per 
week and the WICAT Writing 3-6 product once 
per week. The literal and inferential subtests 
of the Stanford Reading Test were used as pre- 
and post-treatment dependent measures. The 
factors analyzed by mixed ANOVA included two 
between factors, treatment (RO, RW) and gender 
(female, male); and two within factors, test 
(pretest, posttest) and subtest (literal. 
Inferential). The main efTects and most of the 
Interactions were non-significant. However, 
the two factors test. F(l. 100) = 5.926. p = 0.017. 
and subtest. F(l. 100) = 7. 107. p = 0.009. and the 
Interaction of gender and test, F(l, 100) = 6.072. 
p = 0.015. were all significant at the p < .05 
level. The first follow-up analysis examined 
males and females separately using a mixed 
ANOVA which included treatment, test, and 
subtest. The female group obtained no 
significant results for any factor or 
Interaction, indicating that the females were 
not differentially affected by the treatments, 
nor did they experience pretest to posttest gains 
for the test as a whole nor for the literal and 
Inferential subtests. The male group obtained 
significant differences at the p < .05 level for 
the factors test. F(l. 53) - 16. 1 14. p = 0.000. and 
subtest, F(l, 53) = 6.666. p = 0.013. The RO and 
WR treatments were both effective in 
Improving the boys' reading scores from the 
pretest to the posttest. especially for the literal 
portion of the test. In summary, males made 
substantial pre- to post-test gains on both the 
literal (e.s. = 0.43) and Inferential (e.s. = 0.26) 
subtests of the Stanford Reading Test, while the 
females did not (literal e.s. = 0.01; inferential 
e.s. ■ -0.02). The paper concludes with a 
discussion of this troubling finding. 



Introduction 

Extensive computer curricula delivered in 
computer laboratory settings are now 
available. Companies like WICAT Systems. 
Jostens, and the Computer Curriculum 
Corporation are providing thousands of hours 
of systematic courseware products to our 
schools. WICAT Systems alone estimates that 
at least 250.000 students attend WICAT 
Instruction each school day. More and more 
students will receive Instruction to computer 
laboratory settings. 

The research base required for making 
instructional decisions relating to this type of 
extensive computer-assisted Instruction (CAI) 
environments is now being developed, but at 
this time, it is unclear even what questions 
should be asked. For example, what is the 
optimum amount of computer instruction time 
per period and per week? Does this vary by 
ability, interest, gender, or other Individual 
variable? What is the best mix of computer 
delivered subjects? Should computer 
instruction be in lock-step with class 
instruction, or can each progress Individually? 
What role(s) should the teacher play In the 
computer lab? Does the teacher even need to 
come to the computer lab? What sort of reports 
should be generated by these systems? Who 
should receive these reports? Teachers? 
Administrators? The student? Their parents? 
Do these extensive computer environments 
deliver a hidden agenda (Streibel, 1986. 1988)? 

This study addressed a question posed by an 
area principal. Specifically, given that the 
students will attend computer class twice per 
week for twenty minute periods, and given that 
the intent is to increase the students' 
standardized reading scores, then should these 
students receive reading CAI only, or would a 
combination of reading and writing CAI be 
most effective? This study simplistically 
assumes that more is better in CAI. thus the 
reading-only group should outperform the 
reading-and-writing group on the reading 
posttest measure. 

The possibility of gender difference .vas also 
considered because of findings from earlier 
research involving WICAT curicula. For 
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example, in a study involving third grade 
students utilizing the WICAT Systems Primary 
Reading Product, Clariana (1090) reported that 
boys worked significantly faster than girls, e.s. 
= 0.86, thus completing considerably more 
activities over the course of the study. This 
difference occurred even though the girls were 
better readers (e.g., higher standardized reading 
scores). Completing more lessons may 
ultimately translate into higher achievement 
scores. 

A recent review of gender difference in 
computer learning environments determined 
that boys usually do better than girls when the 
computer delivered content is either math or 
science (Signer, 1991). This may be due in part 
to the individualized delivery format of most 
CAI. Differences in girls' and boys' attitudes 
and preferences for working alone or in groups 
has been shown. In a study considering 
individual, cooperative, and competitive CAI, 
Johnson, Johnson, and Stanne (1985) reported 
that girls attitudes compared to the boys were 
adversely affected within the competitive 
condition. Other studies also show that boys 
prefer to work alone while girls prefer to work 
with others (Allen, 1987; Tobin & Garnett, 
1987). This difference may be most evident in 
subjects like math and science, which girls 
traditionally tend to dislike (Esquivel & 
Brenes, 1988; Tamir, 1988; Tobin & Garnett, 
1987). 

When learning alone on microcomputers is 
combined with math or science subject matter, 
this combination of non-preferred delivery 
method and non-preferred content may be to 
much for some students, especially those with 
low-ability, to overcome. For example, Schultz 
and Clariana (1990) examined the gender- 
related learning differences of at-risk eighth 
grade students utilizing WICAT Systems Math 
Products. The students attended computer lab 
for six weeks In groups of twenty-five, Monday 
through Thursday, for two, forty-five minute 
CAI lessons. Two professional teachers (both 
male) were constantly available in the lab to 
assist the students. Extensive progress reports 
were produced and examined each day, and the 
teachers reviewed this with each student 
individually a minimum of twice per week. If a 
student's report revealed a problem, the student 
was given direct help that day. A significant 
gender and ability (i.e., pretest, median split of 
the group) difference was reported. Both below- 
and above-median boys, and also the above- 
the-median girls made significant pre to 
posttest gains on the math achievement 
dependent measure, however the below-the- 



medlan girls did not. The authors concluded 
that the combination of WICAT CAI and 
Intensive teacher-student 
intervention/ interaction was particularly 
beneficial for these at-risk students. H wever, 
even with such close monitoring of students' 
work, the low-ability girls were not successful. 
Post experiment interviews with the two male 
teachers suggests that the low-ability females 
were considerably less vocal than their peers, 
and seemed to "hide-out" by doing Just enough 
work to avoid teacher itervention. Most of 
the boys were unusually vocal, and demanded 
most of the teachers' time. 

Though upper-elementary boys generally out 
perform girls In mathematics (Fennema & 
Carpenter, 1981; KAEP, 1983), girls are usually 
better readers than boys. Though girls might 
not prefer the traditional individual, non- 
social aspects of CAI, their generally better 
reading ability may overcome this non- 
preference. In other words, it may rer r <<re a 
preference mismatch of both the conten.t and 
the delivery of instruction before a significant 
negative impact on learning occurs. This study 
assumes that when the CAI content is reading, 
there should be no posttest performance 
difference between groups of girls and boys. 

In summary, the individual learning 
environment (or lonely learning) associated 
with most extensive computer d elivery systems 
may differentially Impact girls and boys. Girls 
may be more successful in group learning 
situations, which is generally impractical in 
computer laboratory environments. Thus it is 
not computers necessarily, but the way 
corrouters are used, that could exacerbate 
^oiuer differences to the detriment of girls. 

This study considered two pragmatic 
assumptions related to instruction in computer 
labs with extensive amounts of courseware. 
First, doubling the amount of time that 
students spend in a CAI reading curriculum 
will substantially improve their standardized 
reading scores. Second, because the CAI 
content Is reading (Signer, 1991), no difference 
is expected between the girls and boys posttest 
performances. 

Method 

Subjects and Design. 

The subjects involved in this study were fifth 
and w>lxth grade students from an elementary 
school associated with a large southern 
university. The school is a public school with a 
charter to enroll students from the families of 
university employees first. Additional 
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vacancies are filled based on residential 
location. Students living near the school 
receive a higher priority. Additionally, the 
school intentionally enrolls a diverse student 
population which includes many different race 
and ethnic groups and also students from 
varied socio-economic backgrounds. More 
detailed descriptions of the school population 
are available in a dissertation by Jordan 
(1990). 

Pretest data were collected for an Initial pool of 
115 students, which includes most of the fifth 
and sixth grade students in the school. Since 
the lab had only eight stations, students were 
assigned to come to the computer lab in groups 
of eight. The groups were randomly assigned to 
either the "reading only" treatment or to the 
"reading and writing" treatment. The students 
in any group of eight received the same 
treatment. This was necessary so that students 
would not be distracted or upset when they saw 
their peers working on a different curriculum. 

The WICAT curriculum was extensively 
demonstrated to the student's teachers before 
the study began. The teachers were pleased at 
the chance to send their students to the 
computer lab. During f he study, the teachers 
were invited to come observe their students, but 
none came. Computer progress reports were 
sent to the teachers at each six-weeks period, 
however the teachers never commented tc the 
researchers or the coordinator about either the 
reports or their students' progress in the 
computer lab. 

Students attended computer lab twice per week 
for twenty-minute periods and were 
accompanied by a parent volunteer, by a 
teachers aid, or by a graduate student . ern. 
Occasionally, students would be unruly and 
were sent back to their classroom. Though the 
volunteer parents and the interns were present 
In the lab, they lacked the skills necessary to 
maintain the learning environment or give 
positive affective feedback to these elementary 
children. However, most students liked to 
come to the lab and were busy during most of 
the sessions. After 10 weeks, posttest data were 
collected. Students continued to work in the 
lab but no further data were collected. 

Lesson Descriptions. 

The reading only treatment received the WICAT 
Reading Comprehension product only. These 
lessons are designed to enhance reading skills 
like: identifying key ideas in passages, 
identifying fact from opinion, reading and 
interpreting charts and graphs, making 



inferences, and recognizing and deleting 
statements that do not relate to reading 
passages. The WICAT reading curriculum 
consists of several hundred stories ranging in 
grade level from 2nd grade through 10th grade, 
which are grouped into three major categories 
called protocols. The three protocols used 
during this study were inference, deletion, and 
graphing. The students received a mix of all 
three protocols during the course of the study. 

The inference protocol required the student to 
read a story from 10 to 20 screens long. 
Previous pages could be viewed by a simple 
backpage command at any time. A series of 
inferential questions were asked with each 
story portion. The student responded to the 
question and then underlined key words in the 
text that supports their answer. Feedback was 
given after each episode of responding. 
Feedback included knowledge of the correct 
response and also the highlighting of the 
Important words In context by using Inverse 
shading of the words and the correct multiple 
choice response. Students were awarded points 
both for answering the question correctly and 
also for underlining the correct words. 

The deletion protocol required the student to 
read a story from 6 to 10 screens long. The 
students were required to select the sentences 
that did not belong. These sentences were 
crossed out by use of the space bar and the "x" 
key. Feedback included knowledge of correct 
response as well as explanatory feedback after 
each student response. Students were awarded 
points for deleting all the unnecessary 
sentences, however there was a penalty for 
removing correct sentences. 

The graphing protocol required the student to 
view a visual and answer questions about the 
visual. Students were presented with a picture, 
line drawing, chart, or table. About six highly 
specific questions about the visual were asked. 
Students responded to each question by moving 
the cursor icon to the correct location on the 
visual. The correct response was highlighted 
with inverse color. When the student was 
incorrect, the same question would be 
rephi* *ed and a hint would be given. The 
correct answer was given after the second try 
regardless of the student's response. Students 
were awarded points for correct responses on 
their first attempt and partial credit for correct 
responses on their second attempt. 

These reading passages covered a range of 
student reading grade and interest levels from 
about grade two through grade eight. Students 
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were evaluated by the online WICAT Cloze test 
and placed Into the computer curriculum at 
their appropriate level. Clarlana (1991), in a 
study involving 101 upper-elementary school 
students, reported the relationship between 
this WICAT Cloze test battery and a well known 
national standardized reading test, the Iowa 
Test of Basic Skills, as r = 0.83. He concluded 
that the WICAT Cloze test was doing a good Job 
of placing students into the WICAT Reading 
Comprehension curriculum. 

The WICAT system monitors student progress 
and assigns students up or down in the 
curriculum based upon the student's 
accumulated performance. The ability of the 
WICAT system to monitor continuing student 
progress and automatically move a student to 
the correct lesson sequence has also been 
investigated. Bond and Clarlana (1989) 
compared the WICAT computer- adaptive 
placement to manual controlled placement, 
where the experimenters monitored student 
progress biweekly and moved stt'ients to 
different lessons based on a predetermined 
algorithm. No significant differences were 
shown, which may suggest that the computer- 
adaptive system is, at least, no worse than the 
manual process used in their study. 

The WICAT Writing product provides four 
different forms of activities that allows the 
students to compose stories that are either 
informational, humorous, creative, or 
descriptive of a visual that the students 
compose from a menu of "parts". These 
activities are very popular with students of this 
age, and students often request that they be 
allowed to do the writing activities. During the 
writing activities, the students were given 
information as text and questions, were shown 
pictures, figures, and animations, and were 
asked to write a story about what they had seen 
and read. The stories were composed on the 
word processor and were printed on paper and 
given to the student. 

Instrumentation and Analysis. 
Alternate versions of the Stanford Reading 
Test were used to obtain pretest and posttest 
data. The tests were administered under 
controlled conditions in the students' 
classroom by a graduate student and by the 
students' homeroom teacher. The importance 
of doing well on the test was emphasized. The 
tests were composed of literal and inferential 
subtests. The major analys e consisted of a 2 
(treatment: reading only versus reading and 
writing) x 2 (gender) x 2 (test: pretest and 
posttest) x 2 (subtest: literal and Inferential) 



mixed ANOVA on Stanford Reading Test scoree. 
Results 

The means and standard deviations for each 
treatment group are shown in Table 1. The 
analysis of variance summary information is 
shown In Table 2. The two factors test, Ptl. 100) 
= 5.926, p = 0.017, and subtest, Ptl. 100) = 7.107, 
p * 0.009, and the Interaction of gender and 
test, Ptl. 100) = 6.072, p = 0.015, were shown to 
be significant at the p < .05 level. Because of the 
gender interaction, the first follow-up analysis 
examined males and females separately using a 
mixed ANOVA which included treatment, test, 
and subtest. The female group obtained no 
significant results for any of the main effects 
or interactions, indicating that the females 
were not differentially affected by the 
treatments, nor did they experience pre to post 
test gains for the test as a whole, nor for the 
literal and Inferential subtests. The male 
group ob^Jned significant differences at the p < 
.05 level for the factors test, F(l, 53) = 16. 1 14, p 
= 0.000, and subtest, F(l, 53) = 6.666, p = 0.013. 
Figure 1 attempts to display the significant 
interactions noted above. Examination of 
means revealed that the boys made substantial 
pre- to post-test gains on both the literal (e.s. = 
0.43) and inferential (e.s. - 0.26) subtests of the 
Stanford Reading Test, while the girls did not 
(literal e.s. = 0.01; inferential e.s. = -0.02). 

Discussion 

The assumption that "more reading CAI is 
better" was not confirmed. The posttest scores 
of the reading and writing group (mean = 658.5) 
compared to the reading only group (mean = 
657.5) were not significantly different. 
Clarlana (1991) noted that, "At this time, 
computer-assisted Instruction means reading. 
The lesson content may be math, science, 
social studies and so on, but no matter what, 
reading, specifically, reading computer frames 
that have generally low resolution and limited 
surface area, is critical for lnstruction."(p. 107) 

In this study, the students that received the CAI 
writing curriculum were required to read, 
follow directions, and understand in order to 
be able to do the "fun" writing activities. 
Though the curriculum Involved writing, It 
may have affected the students' reading skills. 
If so, it is possible that any type of CAI content 
may positively impact reading skills. 

The assumption that there would be no gender- 
related differences in this CAI reading 
curriculum was also not confirmed. 
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TABLE 1 

Means and standard deviations. 



Females 
Reading only 
(n=23) 

Reading + writing 
(n=26) 

Males 

Reading only 
(n=26) 

Reading + writing 
(n=29) 



pre Literal pre Inferential 



648.3 
sd64.5 

665.1 
sd79.2 



627.5 
sd58.0 

649.2 
sd74.5 



648.9 
sd69.4 

654.7 
sd80.3 



629.7 
sd66.7 

641.9 
sd77.8 



661.5 
sd80.2 

653.9 
sd83.9 



665.8 
sd70.2 

669.5 
sd68.8 



post Literal post Inferential 

647.2 
sd78.2 



653.8 
sd79.7 



654.6 
sd58.3 

655.5 
sd56.1 



TABLE 2 

ANOVA summary table. 



.sourc 



ss 



df 



Between Subjects 

Effects 

Treatment (A) 

Gender (B) 

AxB 

error 

Within Subjects 
Effects 

Pre/post tests (C) 

AxC 

BxC 

AxBxC 

error 

Llt/Inf subtests (D) 

AxD 

BxD 

AxBxD 

error 

Interaction (C x D) 

AxCxD 

BxCxD 

AxBxCxD 

error 



5815.846 1 

2539.659 1 

473.810 1 

1647215.569 100 



ms 



5815.846 
2539.659 
473.810 
16472.156 



0.353 
0.154 
0.029 



0.554 
0.695 
0.866 



15050.998 




15050.998 


5.926 


0.017* 


4510.782 




4510.782 


1.776 


0.186 


15423.388 




15423.388 


6.072 


0.015 • 


50.432 




50.432 


0.020 


0.888 


253992.349 


100 


2539.923 




4770.428 




4770.428 


7.107 


0.009 •* 


132.191 




132.191 


0.197 


0.658 


57.582 




57.582 


0.086 


0.770 


384.844 




384.844 


0.573 


0.451 


67127.272 


100 


671.273 




985.877 




985.877 


0.895 


0.346 


1632.693 




1632.693 


1.483 


0.226 


384.120 




384.120 


0.349 


0.556 


556.034 




556.034 


0.505 


0.479 


110124.155 


100 


1101.242 





F^YiV ^ e , graph of . the pretest t0 P° stest changes for males (dashed lines) compared to females 
(solid line) broken out by literal and inferential subtests. 
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Examination of means revealed that the males 
made substantial pretest to posttest gains on 
both the literal (e.s. = 0.43) and inferential (e.s. 
= 0.26) subtests, while the females did not 
(literal e.s. = 0.01; Inferential e.s. = -0.02). 

Are there gender-related learning differences 
in computer lab environments? Schlmmel 
(1986) stated "If feedback operates as a kind of 
assistance, then feedback may be more 
frequently sought and more effectively used by 
females than males" (p. 113). Perhaps girls 
take more advantage of help screens and other 
aspects of "good" computer lessons than boys, 
thus girls may actually read more Information 
than boys during a lesson, and take longer to 
complete the lesson. At the same time, the 
chunk size of CAI tends to be small. There may 
be little advantage in using the extra available 
information provided by help screens. It 
seems that computer lab work will tend to 
favor learners that are aggressive or hurried 
(Clarlana & Smith, 1988) and may punish 
careful or thoughtful learners. This has been 
referred to as a "production mentality" 
(Streibel, 1986, 1988), since students are 
rewarded for completing more activities. 
Quantity wins over quality. Such an 
environment may favor boys over girls. 

Importantly In this study, teachers were not 
available to give praise or note student's good 
work. This may be a critical aspect of CAI 
work, especially with young children. Peterson 
and Fennema (1985) found that the 
mathematics achievement of fourth grade 
females was positively correlated with the 
amount of personal assistance they obtained 
and the opposite was true for males. Clarlana 
(1990b) Investigated the effects of teacher style 
on rate of CAI reading lesson completion by 
first grade students. An authorative teacher 
style resulted in significantly faster lesson 
completion compared to a laissez-faire teacher 
style. Teachers' style is an often overlooked 
variable In CAI research. 

Teachers' presence may impact students' 
motivation to work. In thJ i study, perhaps the 
boys received intrinsic motivation when 
working on the computer while the girls did 
not. If the teacher had been present, the girls 
motivation, which may focus on their teacher, 
would have been higher. This begs some 
important pragmatic questions, "Should 
teachers be present in computer labs?" and 
"What sort of role should the teacher assume In 
the computer lab"? 

In summary, good instructional practice in 



computer labs with extensive curricula depends 
upon an adequate research base. The 
simplistic model that "more is better" is not 
adequate. Apparently, a computer lab is a 
complex learning environment that requires 
considerable examination, both qualitative 
and quantitative. More research and better 
models are required concerning the learning 
environment associated with CAI labs. 
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THE INTEGRATIVE TEACHING MODEL 



Carolyn M. Massie, University of Virginia 
Larry G. Vblfc, University of Virginia 



Abstract 

This paper will explain how one Information 
processing model of teaching can merge with 
computer technology to create a more effective 
data retrieval process. The hypermedia 
program on the Macintosh computer, 
HyperCard, will be used to help focus students' 
attention on the material being presented In 
the lesson. In addition to the hypermedia 
program being able to isolate specific data 
display Information, using animation and 
speech can also Increase students 1 on-task 
behavior. 

The Integrative Teaching Model combines 
content information with inductive and 
deductive reasoning to help develop students' 
thinking skills by having students progress 
through five instructional phases. This paper 
will summarize the theory behind the model, 
give reasons for using an information 
processing model, and will point out the 
advantages of using HyperCard for the data 
display and retrieval. The Integrative Model's 
five phases are described within a sample 
lesson plan. References will be included which 
would enable an educator to design hypermedia 
stacks on HyperCard. 

An Instructional Merger 
One of the major criticisms voiced against our 
educational systems has been the low level of 
thinking that teachers have required from 
students. Too frequently, our classroom 
requirements have not extended beyond 
memorization and recall of facts and/or 
figures. This mere accumulation of descriptive 
knowledge does not prepare our students to 
critically interpret data. 

Some instructional strategies have been 
developed and tested during the past few 
decades which encourage students to process 
information. Using information processing 
teaching strategies have been fo'ind to develop 
students' thinking skills (Steinberg, 1985; 
Rosenshlne & Stevens, 1986). The information 
processing teaching model discussed here was 
first conceptualized by Hilda Taba (1965, 1967) 
and later described by Eggen and Kauchak 
(1988) as the Integrative Model. Eggen and 



Kauchak'* conceptualization of the model 
includes five phases which combine inductive 
and deductive skills with content. 

As computer applications become more 
powerful and more available to teachers, 
educators who feel comfortable experimenting 
with computer programs will discover ever 
Increasing ways of using computer technology 
which will complement instruction. One such 
modification will be described in this paper. 
The hypermedia application, HyperCard, found 
on the Macintosh can be used in conjunction 
with the Integrative Teaching Model to make 
better data displays. Being able to reveal only 
that data which the teacher wants the students 
to respond to can increase on-task behavior 
during the lesson. Instead of students looking 
at immobile text or pictures, a HyperCard stack 
can be designed with objects which move, 
pictures which talk, and music. Other 
advantages of using a HyperCard stack with the 
model will be explored. 

In order to ensure that the reader will be able to 
understand the theory behind the model's 
development, background for the Integrative 
Model will be given in this paper although the 
conference session only touched on it briefly. 
This paper will also summarize the 
instructional benefits of using the Macintosh 
HyperCard application for the data display and 
retrieval. The phases of the Integrative Model 
will be defined in a lesson plan which will lead 
the reader through both the five phases of the 
model and the merger of a HyperCard 
application. 

Hilda Tiba't Strategy For Interpreting Data 

Several decades ago, Hilda Taba (1965) 
challenged the traditional method of teaching 
in the area of social sciences. The traditional 
approach was to have students memorize vast 
amounts of descriptive data for recall on 
objective tests. Taba noted that the twentieth 
century knowledge explosion has caused many 
facts to be obsolete by the time students have 
mastered them. Rather than burdening the 
memory with volumes of descriptive 
knowledge, Taba felt that teachers should 
instead help students develop organizing 
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conceptual schemes. 

Taba built a whole social studies curriculum 
around these theories in the Contra Costra 
School District and popularized the 
terminology, teaching strategies (Joyce & Well, 
1986). She believed that certain cognitive tasks 
such as "concept formation, interpreting data 
and making inferences from them, and the 
tasks of applying learned generalizations and 
facts for purposes of hypothesizing, predicting 
possibilities, and explaining new phenomena 
could be taught at a fairly early age" (Taba, 
1965, p. 468). In order for students to learn 
these skills, however, teaching strategies had 
to be used which were specifically designed for 
developing these cognitive tasks. Taba 
developed three teaching strategies which were 
designed to induce growth in these cognitive 
tasks. 

The teaching strategy designed for 
"Interpreting data" was built around three 
mental operations Taba referred to as 
interpreting, inferring, and generalizing (Joyce 
* Well, 1986). The students' three mental 
rations were labeled "covert activities." The 
uie activities children engaged in were 
- led "overt activities." The eliciting 
questions teachers were to ask formed an 
Integral part of the information processing 
strategy: the teacher's questions were to help 
guide and stimulate the students' thinking ( 
covert activities) about the content found in the 
overt activities. Taba emphasized that teachers 
needed to organize instruction in two 
dimensions at once when they planned a lesson 
or unit: one level represented significant 
content, and the other level addressed the 
learner's cognitive skills for processing the 
content (Taba, 1965, 1967). 

To help distinguish which content was 
significant, the social studies curriculum 
content was organized into hierarchial 
concepts. Taba's developmental learning 
curriculum was heavily based on Piaget's 
theory about thinking and Bruner's beliefs 
about the n ature of concepts (Taba, 1965: Joyce 
& Weil, 1986). Taba believed concepts should be 
presented through learning sequences which 
would gradually move the student from what is 
already known to more abstract and complex 
ideas. As students progressed through these 
sequences, engaged in interpretation, and saw 
relationships between concepts, they would be 
able to go beyond the given information to 
make inferences and eventually could make 
generalizations. Eggen and Kauchak (1988) 
used Taba's teaching strategy for interpreting 



data to form the basis their Integrative 
Teaching Model. 

The integrative Model 

The Integrative Teaching Model has five 
Interrelated phases which encourage and 
expand students' thinking skills about the 
concept being taught. By (1) describing, (2) 
comparing and contrasting, (3) explaining, (4) 
making hypotheses, and (5) forming 
generalizations, the students move beyond just 
attaining factual knowledge (Eggen & Kauchak 
(1988). This model challenges students to bring 
together inductive skills, deductive skills, and 
content. The steps in this model will be more 
fully described In the lesson plan that follows 
later In this paper. 

Content which matches this model well can 
usually be displayed graphically. Using visual 
organizers help students develop the cognitive 
organizational schema that Taba believed so 
lmpor int. In fact, Taba included various 
grapaic examples which illustrated content 
organisational schema. These visual 
organizers Included spirals, grids, and circles 
with radiating lines (Taba, 1965). These 
schema displayed data which could be used in 
the lesson's overt activities. The data displays 
became an indispensable part of Eggen and 
Kauchak's Integrative Model. 

Data Retrieval 

The data displays presented by Eggen and 
Kauchak (1988) include charts, maps, and 
graphs. The most frequently used method of 
display Is the chart. Charts usually have a 
matrix which contains cells. The first step of 
the Integrative Model has students describe the 
information in an Individual cell. Without the 
advantage that HyperCard can offer, students' 
attention cannot be focused as easily on the 
individual cell. Instead, the entire matrix is 
visible. Some teachers have tried to solve this 
problem by putting the data on the chalkboard 
as It is being presented. The disadvantages to 
this method are the management problems 
that can occur when the teacher turns around 
to enter the information and the interruption 
In the flow of verbal interaction that occurs 
when the data is being recorded. Using either 
the chalkboard or having an entire matrix 
displayed can prevent students from focusing 
solely on the data being discussed and can 
cause distinct problems for attention deficit 
children. Using the HyperCard application 
allows the teacher to "call up" information on 
the matrix as the lesson progresses. The 
Macintosh HyperCard computer application 
can enhance the Integrative Model's data 



display and retrieval process. 

A HyperCard stack works much like 
transparency overlays for the overhead. Each 
"card" In the stack can build on the previous 
layer or cards. When the teacher uses 
HyperCard to present the matrix, the students 
see only those cells which have been discussed 
to that point In the lesson. This allows the 
teacher to be more confident that students are 
concentrating on the particular data being 
discussed. The students' attention is not 
"jumping ahead" to other cells on the chart. 
HyperCard also has the ability to go directly to 
any card in the stack that the teacher might 
choose to discuss next or to review. 

HyperCard has the power to increase on-task 
behavior in another way. The stack can be 
animated, given sound, and can present 
material from f. variety of sources such as 
textbooks, magazines, photographs, and 
records. All of these have the potential to 
capture the child's attention during the lesson 
and to make the data more meaningful. 

The teacher can prepare the HyperCard stack 
ahead of time and can store it easily for future 
use. The storage room is minimal, and 
although the "chart" has been completed, it can 
be altered easily. This can be a real 
convenience if the teacher finds other material 
that could be included; or if after the lesson is 
taught once, the teacher finds it would be better 
to change some of the data. 

A final reason to teach the Integrative Model 
with a HyperCard application concerns the 
reinforcement possibilities. After the lesson is 
finished, the teacher has the option of making 
a copy of the disk with additional directions or 
information that students can use 
independently for review of the concept. This 
independently used disk could also be 
beneficial if used with students who were 
absent during the classroom presentation. 

The following sample lesson plan will 
demonstrate how teachers can use the 
Integrative Model with a Macintosh HyperCard 
stack. It will provide suggestions for merging 
HyperCard with content that is displayed on a 
matrix. 

Seasons: A Lesson Plan 

Grades: 1-2 

Pre-Computer Activity 
Decisions: 

The first decision the teacher must make is 



whether the content matches the teaching 
strategy. If it does, during the lesson 
preparation, the teacher needs to decide what 
information needs to be Included on the 
matrix. The most "processable" type of data is 
that which is factual; the next best type Is a set 
of narrow generalizations or a mixture of facts 
and narrow generalizations (E ggen & Kauchak, 
1988. p. 177). The teacher should then sketch or 
copy (from a reference book) a matrix data 
display which serves to help organize the 
lesson's content. This data display should be 
conceptually comparable to a set of index 
cards. Cards should be numbered so they can be 
linked to one another to the HyperCard stack. 
The teacher will then customize a duplication 
of the set of " index cards" on the Macintosh 
HyperCard program. 

If possible, especially with young children, the 
data should be entered on the matrix so that the 
lesson begins by looking at the cell to the upper 
left hand corner. Simple directions should be 
given that alert the teacher or student how to 
progress through the stack. 

The teacher should write the accompanying 
lesson plan either before the stack is designed 
or do it as the stack is being drawn on the rough 
draft index cards. The lesson plan should 
include all the five Integrative Model phases 
and should contain all the eliciting questions 
that the teacher plans to ask. 

The matrix should then be entered on the 
HyperCard application. Two references are 
given in the materials section of this lesson 
plan which could help the teachei' who is 
unfamiliar with the HyperCard application. 
When the lesson is taught, the teacher should 
use a data projection pad for students' easy 
viewing. 

Objectives: 

Students will be able to describe the four 
seasons, compare and contrast their 
similarities and differences, make an 
inference that goes beyond the data presented 
but Is related to it, and make generalizations. 

Materials: 

Macintosh computer, monitor, data projection 
pad, screen, and lesson plan with a completed 
stack on a disk. 

Bull, G. & Harris, J. (to press). HYPERCARD 
FOR EDUCATORS: INTRODUCTION FOR 
BEGINNERS. Eugene, Oregon: International 
Society for Technology to Education. 
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Eggen, P. and Kauchak. D. (1988). STRAGEGIES 
FOR TEACHERS: TEACHING CONTENT 

AND THINKING SKILLS. Englewood Cliffs, 
New Jersey: Prentice Hall. 

Hoftneister, J. F. and Rudowskl, J. B. (1989). 
HYPERCARD IN EDUCATION: SURVIVAL 
NOTES. Cincinnati, Ohio: Apple Computer. 

Computer Activity 
Procedures: 

Anticipatory Set. Today we are going to learn 
something about each of the four seasons of the 
year. The names of the four seasons are listed 
on this side of the chart. (The teacher would 
call on a child to read the names of the four 
seasons that are displayed in the left column of 
the chart.) Across the top of this chart we will 
find the words that tell about what we are going 
to learn about the seasons (the teacher or a 
child will read the words, "weather, holidays, 
and food"). 

(The teacher will then click the mouse on the 
first cell in the matrix that the class will 
describe. Each cell will have not only a picture 
to be described on the top matrix layer, but 
after the mouse is clicked, additional cards will 
"pop up" with related data. For example, the top 
picture beside the word Winter is a group of 
snowflakes. Underneath is an animated group 
of cards that show a house with snow around it 
and a person who tries to ski. When the skier 
falls, he says, "Oh, dear me!") 

1. Description. (Students are asked to describe 
the data In one cell on the matrix.) "There are 
some empty boxes on this chart: as we learn 
about seasons, different pictures are going to 
*pop up' for you to discuss." The teacher then 
points to the picture that is in the cell beside 
Winter and under Weather. The teacher might 
then say," Look at the weather In winter. What 
do you notice here?" Then the teacher would 
click the mouse to display the cell underneath 
the top cell's card. "What do you see in this 
picture? What is the man in the picture doing?' 
After completing the description of the 
information in the first cell, the teacher moves 
on and has students describe a second cell. 
"Now let's look at the picture beside the word 
Summer under Weather. What do you see?' (The 
teacher would let them describe the picture of 
the sun. Then the mouse would be clicked to 
reveal the card underneath the top cell's card 
which is a picture of a lady with a swimming 
suit on.) "What can you tell me about this 
picture?" 



The teacher can make the decision about how 
quickly the students can move to Phase 2. The 
students might be asked to describe all the cells 
on the matrix before moving to Phase 2. The 
teacher may choose to have the students 
describe all the cells in one column or row, or 
they may begin to compare the two cells they 
have Just described. Students may 
unconsciously move to higher level phases 
when they are asked to describe the data, and 
they should be allowed to do this if it happens 
naturally. 

2. Comparing. (Students are asked to compare 
two or more cells. The teacher facilitates the 
student moving through this phase by asking 
probing questions.) Comparing helps students 
structure the information. This phase like the 
first one, is inductive. The teacher could ask, 
"How would you compare the weather In the 
winter to the weather in the summer?" 

3. The third phase, Explanation, develops 
students' critical thinking skills. Students are 
asked to explain why the similarities or 
differences existed in Phase 2. Children can 
document their answers with information that 
can be found on the chart. Forming 
explanations in this manner is a deductive 
process. 

Eggen and Kauchak (1988) noted that while 
summarizing and generalization can occur 
within a discussion about the data in one cell, 
children should be cautioned about 
overgenerallzing. They stated that 
overgenerallzation is one form of conclusion 
assessment, a fundamental critical thinking 
skill (p. 192). 

4. Hypothesizing is the fourth phase. Students 
are asked a hypothetical question which is 
directly related to the Information they have 
been using In the first three phases. Students 
are asked to use information on the chart to 
support their reasoning. For Instance, "If the 
weather suddenly became as hot In the middle 
of the winter as It is in the summer, what could 
happen?" The hypothetical question should be 
directly related to the information that has 
Just been processed. Hypothesizing is another 
deductive thinking skill. 

5. The final phase Is Generalizing. Students 
summarize the content by formulating one or 
more generalizations. By the time this phase is 
reached, certain patterns in the matrix will 
have been discussed: the generalization will 
probably describe these patterns. Sometimes 
only one generalization is appropriate while 
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other lessons may support several 
generalizations. A child might make a 
generalization about the Weather column on 
the Seasons chart after all four cells have been 
discussed such as, "People do different things 
for fun when it's warm than when it's cold." As 
children disagree about each others' 
generalizations and expand on them, 
discussion in the classroom can become lively. 

These five phases can be repeated several times 
during one lesson as the teacher leads the class 
through the different cells in the matrix. How 
much data the children describe before they 
move to the other phases may depend upon the 
information in the cell and the age of the 
children. 

Post-Computer Activity 

Independent Work. "Now, I am going to hand 
you a large piece of newsprint. I want you to 
fold it in half; then fold it in half again. I am 
going to put my folded piece of newsprint on the 
bulletin board so you can see what I am going to 
do next. I am g oing to number my boxes 1, 2, 3. 
and 4. (The teacher should let them number 
their quadrants and should circulate around 
the room to see if they have done this.) Next, I 
want you to draw something In your number 1 
box that you learned about winter." (Proceed by 
giving directions for each numbered section 
and season.) 

Closure: 

'Today, children, we have discussed the four 
seasons of the year. Tomorrow you will work in 
small groups and will make dioramas for each 
season. Who can tell me what a diorama is? We 
will use large shoe boxes for our dioramas. I 
have collected some items that you could use in 
your dioramas, and you can bring in small 
things from home that can be used." 

Evaluation: 

During the lesson, the teacher will assess the 
children's responses (errors will be corrected 
through probing and giving clues). The teacher 
will informally check the students' drawings 
on the newsprint for accuracy. This summative 
evaluation will allow the teacher to correct any 
misconceptions the next day before the next 
seasonal activity progresses. 

Follow-Up: 

Reinforcement can be aided by leaving a copy of 
the disk, Seasons, available at the computer 
station. Extra narrative information may be 
added to this disk. Some of the children's own 
descriptive statements and generalizations 
might be very appropriate. 



Summary 

The Integrative Model of instruction can help 
develop students' thinking skills as they 
process information. By using the HyperCard 
application for the data display and retrieval, 
computer technology can enhance the 
instructional delivery of this teaching model. 
The main advantage HyperCard has to offer is 
related to the improved focus and attention of 
students during the data retrieval part of the 
lesson. The teacher can also appreciate the 
subsequent data modification capabilities that 
HyperCard affords and the easy material 
storage of the charts, maps, or graphs. 
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TWO DISTINCT CONCEPTS OP GROUPWARE 



Jerome A. Popp, Southern Illinois University 



Abstract 

Research on team-study has produced the 
outlines of a theory of classroom qualitative 
structure. When these beginning ideas are 
further developed, it can be seen that there are 
two distinct concepts of groupware. The nature 
of groupware required to support centralized 
classrooms and team-study classrooms are 
developed by means of the qualitative theory. 

Introduction. 

The concept of groupware has emerged from the 
work of instructional designers and is now the 
focus of many discussions. One way to advance 
the development of this general notion Is to 
place it within an appropriate theoretical 
context. In what follows, I will outline the 
features of a comprehensive theory of 
classroom organization, and then use this 
theory to investigate the nature of groupware 
and its place within instructional 
methodology. The following analysis reveals 
that there are two distinct concepts of 
groupware, and it is important for 
instructional designers to understand the 
nature of this distinction and its theoretical 
underpinnings. 

Classroom Qualitative Structure. Every area of 
research requires a basic account of what there 
is to be studied. In geology, the theory of plate 
tectonics forms the blueprint or paradigm in 
terms of which observable events are to be 
described and explained. Some refer to such 
accounts as theories or laws of qualitative 
structure (Newell and Simon, 1990). Such 
theories serve as paradigms or blueprints that 
determine what kinds if things offer the best 
explanations of the phenomena under study 
(Popp 1978, and Popp 1980). 

Within the research on teaching, there has 
emerged a qualitative theory that characterizes 
classrooms in terms of a three dimensional 
social structure. These three dimensions 
consist of the following: (i) rules of 
participation that define and govern activities 
of students and teachers, (11) rules of benefit 
accrual that regulate how valuables are 
distributed, and (ill) management structures 
that determine the role of the teacher as the one 
who is in authority within the classroom. 
Every classroom can be characterized by how it 



regulates participation, benefits, and manager 
behavior. 

Rules of participation can be divided into three 
subsets: there are rules that regulate 
competitive activities such as arm wrestling; 
there are rules that regulate cooperative 
activities such as the three-legged-race; and 
thirdly, there are Individual activities that do 
not require other participants. 

The rules of benefit accrual are also divided 
into three sub-forms. A familiar way to 
distribute rewards is to give them to the 
winners of competitive activities, eg., curve 
grading practices. In addition to competitive 
benefit rules there are collective distributions 
that give each participant an equal share of the 
benefits. An individually structured benefit 
distribution system is exemplified by the 
grading contract. 

Following Ouchi (1984), one can view the 
management of classrooms as being of three 
types. Some organizations are managed by the 
"holding company" approach where each unit 
within the organization functions as an 
autonomous unit. In traditional classroom 
theory this Would be referred to as 
"permissivism." In a highly "centralized" 
organization, all decisions are made or 
overseen by a central authority. In classrooms, 
we typically think of this management form as 
authoritarian. Finally, there is the 
management structure that many refer to as 
"participatory." In this form, the participants 
in the organization are constantly negotiating 
who will be the bearers of what kind of 
authority and responsibility. 

From an investigation of the various 
combinations of how these three dimensions 
are instituted within a classroom, one finds 
that certain combinations are workable and 
some that are not. It turns out. according to this 
line of logic, there are three basic modes of 
instruction that are workable. Traditional 
teacher-centered instruction uses competitive 
rules of participation, a competitive benefit 
structure (curve grading), and a highly 
centralized-authoritarlan management form. 
Individualized instruction is characterized by 
individual rules of participation and benefit 
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distribution, and a laissez-faire management 
structure. Team study is based on cooperative 
participation, collective rewards, and a more 
participative management form, eg., the 
decision as to what will be studied can be 
shifted from the teachers to the students in 
many cases (Slavln 1983). 

Each of these three approaches to classroom 
organization demands different behaviors 
from teachers and students. Of particular 
interest to instructional designers Is the 
finding that the type of practice exercises and 
study activities used by each of the three are 
quite different from each other. For example, 
in team study, it is Impossible to use the 
questions at the end of chapters as team 
practice. End of chapter questions seem to 
assume that the user is working alone, whereas 
practice for team study must: be constructed so 
as to support group discussions. The exact 
nature of the best team-study practice- 
questions is as yet unclear, but it does seem 
that the questions must be more challenging 
than those questions effective for individual 
students working alone. 

From my own investigations, it seems that 
when questions developed for individualized 
study are used in a team context, the quality of 
group discussion is very poor and students do 
not progress through the material as quickly as 
they do with group questions. On the other 
hand. If effective team-study questions are used 
for Individualized instruction, then students 
quickly become frustrated (Popp unpublished 
a). 

The questions required to support lively group 
discussions must be open-ended enough to 
induce an investigation of meanings and 
interpretations. Students are required to 
attempt to understand each other. I think that 
through these efforts at serious 
communication students develop the increased 
abilities to take the other person's point of 
vlew-a finding that Slavln (1983) Is fond of 
pointing out. There is no doubt In any 
educational researcher's mind that the team- 
study classrooms do produce higher levels 
social Interactional skills than do alternative 
designs. 

For each of these three instructional forms, we 
can ask about the best or most effective use of 
the micro-electronic technology. Since these 
three Instructional forms are theoretically 
different, one would expect that the role of 
computers within each form should be 
different, one from another. Moreover, as one 
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evaluates the potential for computers in each of 
the Instructional forms, one should note that 
the computer must be introduced in a fashion 
that does not undermine the social 
organizational pattern for that form. In other 
words, Just as all mixtures of the various types 
of qualitative structure do not fit well together, 
all applications of computers in classrooms 
will not readily articulate with all classroom 
structures. 

Individualized Instruction. In a sense we need 
not consider the case of individualized 
instruction because the notion of 
"individualized groupware" is an oxymoron. 
However, there is an important point that 
results as a kind of theorem deriving from the 
theory of classroom qualitative structure. The 
computer allowed for the development of a very 
sophisticated form of the earlier hard-copy- 
based "programmed learning." Teachers came 
to value a classroom that could provide a 
computer (or at least a terminal) for each 
student. In many cases, two or three students 
were seated at one computer or terminal; these 
students were said to be engaged in 
"programmed instruction." 

Under the regime of the theory of qualitative 
structure characterized above, these students 
must be acting in accordance with certain rules 
of participation. It Is clear that if it Is not the 
case that two students are simply watching a 
third student who is actually the one engaged in 
the learning process, then the three students 
are a group and have a given pattern of rule- 
governed interaction. 

Furthermore, it is a mistake to think in terms 
of "individualized programs" that allow more 
than one student to use the program at the same 
time. The fact that there are going to be student 
interactions Involved In the very use of the 
program suggests that instructional designers 
must take this student Interaction into account 
when constructing such programs. 
Individualized instruction with kibitzers (UK) 
actually Is no instructional design at all for the 
kibitzing students. 

The following methodological conclusion can 
be die Am. If we are seeking to design academic 
"groupware" that Is truly "designed" for group 
study, then the theory of classroom qualitative 
structure characterized requires Instructional 
designers to decide for each case of groupware 
construction which type of groupware-- 
teacher-centered groupware or team- centered 
groupware they seek to build. 




Teacher-Centered Groupware. In the 

schoolhouse of old, there was a chalkboard 
upon which the teacher wrote, and there were 
individual slates for students. When students 
wrote on their slates, it was difficult for the 
teacher to monitor what each student was 
doing. When the teacher wrote on the 
chalkboard, it was often difficult for every 
student to see what was being written. 

With a classroom LAN system this is no longer 
the case. The teacher can write on each 
students slate. Any given student's "seat work" 
can also be readily monitored by the teacher. 
The hardware is, of course, already in use but 
the groupware driving it is not all that 
sophisticated. What would advance this area of 
research? 

The distinction between automation and 
Implementation is helpful. We could develop an 
intelligent LAN or teacher-centered expert 
system by automating what teachers do. There 
seems to be no limits to what is imaginable in 
this regard. For example, the Lincoln robot at 
Disney World could be further refined to 
become a lecturer on the Civil War. With 
appropriate sensing, the robot could respond to 
the attention level of students. 

If on the other hand, we set out to develop an 
Implemented system, then we do not have to 
enhance or automate what teachers do; we can 
directly pursue the learning of students by 
developing an appropriate expert system. 
Perhaps we should change our terminology 
from "teacher-centered classrooms" to 
"centralized classrooms." In any case, note that 
we seek a groupware solution to student 
learning. T e want student learning to occur in 
a social context. Social skill development is 
always part of our objectives. Thus, we must 
avoid slipping back into thinking of a 
computer managed room filled with instances 
of Individ* lllzed instruction. 

As we pursue the question of the nature of 
teacher-centered or centralized groupware it Is 
Important to understand that the traditional 
classroom practice of curved grading is a 
benefit structure that induces individual 
competition and undermines cooperation. 
Helping other students can decrease one's 
benefits, as every American student knows full 
well. What does this mean for us? 

We could actually enhance the competitive 
nature of the classroom. For example, we might 
engage students in, say, math practice. As each 
student worked her or his problems, in a comer 
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of the video presentation or "slate" there could 
be the current class mean for problems 
successfully solved, and a comparison with 
that individual student's progress. Each 
student would be aware at all times how he or 
she was doing in the competition. 

Of course one can overdo competition; but the 
psychological question I want to raise is a 
different one. Our groupware should be 
psychologically consistent. Note that if we also 
established a system of student E-mail on the 
same LAN, we would be supporting cooperation 
in the sense of allowing students to work 
together. In the non-electronic classroom this 
was forbidden because student io-student 
communication was said to cause disruptions 
of the classroom's academic activity. (Actually 
there is a more important reason to be 
mentioned shortly.) A student E-mail system 
would not have that effect, in all likelihood. 
Nevertheless, we still have the psychological 
and ethical question of establishing a 
competitive benefit structure in the classroom, 
and then also establishing an antagonistic 
system for students to cooperate. 

An even more sinister dimension exists. Robert 
Slavin had described cases of students 
sabotaging each other in order to enhance their 
relative position in the competition of the 
classroom. Around the halls of chemistry on 
every campus, there abound stories of the 
brilliant methods of sabotage used by the pre- 
medlcal students. Do we want to create a system 
that allows unacceptable social devices to be 
created and used by students? Rapid 
informational transfers may greatly increase 
the potency of disinformation. E-mail type 
communications within a competitive 
classroom seem, in many ways, abberrant. 

My investigation of the classroom qualitative 
structure described above has led me to the 
thesis that a teacher-centered classroom must 
be based on a competitive benefit structure 
(Popp, unpublished paper b). The reason that 
student-to-student communication is typically 
proscribed in the conventional classroom is 
the fact that such communication undermines 
to some degree the competitive nature of these 
classroom qualitative structures. I do not want 
to argue this entire thesis here; however, it does 
seem that instructional designers cannot avoid 
the issues of how much competition the 
groupware will impose on students, and how 
much cooperative activity should be allowed or 
provided by the system. Moreover, arbitrary 
decisions in this regard may not always be 
compatible with what we know about the 
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psychology of learning. 

Team-Study Groupware. The team-study 
methodology is based on the notion that there 
should be no Interpersonal competition within 
the classroom. In a feeble attempt to avoid 
using interpersonal competition, the older 
"group" methods of the late 1960s and early 
1970s attempted to avoid the confines imposed 
by teacher-centered instruction. Such attempts 
did not work, to part, because they provided no 
individual accountability, and this led some 
students to simply sit out the lessons. The team 
study approach holds students individually 
accountable while at the same time making 
success and failure a group experience. Each 
student has a personal stake to the results of 
the work of all team members. This aspect of 
the methodology explains much of its success 
(Popp 1987, and Popp 1989). 

Another reason why the earlier group 
approaches did not work stemmed from the 
fear of imposing on students. Students were 
encouraged to set their own goals and to pursue 
them. The immaturity of the students was 
discounted. The lack of academic structuring of 
these approaches led students to create the 
label, "the group grope." The team-study 
methods provide academic structuring of the 
material while still allowing students to make 
study methodology decisions. 

How could the team methodology be enhanced 
by computer capabilities? As I discussed 
previously (Popp 1990), the Laboratory School 
created by John Dewey at the University of 
Chicago at the turn of the centur" attempted to 
develop academic learning around student 
selected activities. One of the problems that all 
such approaches face is that of information 
availability. If students are actively analyzing 
a problem, sooner or later their efforts arc 
going to be stymied by a purely factual 
question. If the information required is not 
available to the students in their classroom, 
then students must ask the teacher-who may 
or may not have it; if the teacher's knowledge 
must be imported, then there is a loss of student 
autonomy that is directly felt by the students. 
The other alternative is to stop the inquiry, for 
example to go to the library. The difllculty here 
is that when immature scholars are delayed or 
inhibited to any way, they quickly lose their 
eagerness to solve the problem. 

If there were available to each team, a 
database-or better still an expert system--to 
which students could appeal when purely 
informational questions were encountered, 
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students could experience greater analytic 
powers; in fact, it may well be the case that 
many students have never experienced what it 
means to solve a difficult problem after 
considerable effort. Just as students have to 
learn the quaint ways of the libraries, they 
would have to learn how information is 
organized, stored, and recovered. One 
dimension of the curriculum would be cemtog 
to know one's way around the world's 
information. 

One can imagine a developmental approach to 
learning about information and expert 
systems. In the primary grades, the databases 
and expert systems would be structured in ways 
appropriate to the students. In later grades, the 
complexity of the systems would Increase, as 
would the amount and kind of "help" screens 
and tutorials. Instructional designers would 
create products to bridge the gap between the 
primary grades and the expert systems in use 
by the professions. 

The instructional design of team study lessons 
to this case would require the integration of the 
topics studied and the form of the information 
to the database being used by the teams. The 
best organization for students could be 
explored as related to the various grade levels. 
In any case, by the high school years, all 
students should be using typical forms. What is 
interesting about this role of the computer is 
the fact that hardware, software, and hard copy 
must be integrated so as to produce a workable 
classroom system. 

Groupware could enhance the team-study 
approach beyond the providing of improved 
informational and expert resources. In the 
current team-study classrooms, students are 
given hard copy lessons or modules. These 
lessons could be written to an electronic form. 
That is, the computer could present the 
expository material of the lessons. This poses 
an interesting problem. The team-study 
approach does not make use of lecture or 
teacher presentations to any great extent. If we 
computerize the lessons, aie we not sliding 
back into a teacher-centered classroom? 

With auditory input capability, the members of 
the team could interact with the electronic 
lesson the way they interact with each other. 
Only one person can talk at a time if anyone is 
to be understood. So, the rules of participation 
that Involve Interaction with a computer create 
a regime that is not all that different from the 
one currently to use by team-study classrooms. 
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But there Is a danger of our writing groupware 
that allows the computer to become tfte source 
of all wisdom- -In effect, a lecturer-questioner. 
We do not want the students to talk to the 
computer but to each other. To avoid the 
slipping back into teacher-centered thinking 
we must be clear that the students have and 
should retain the autonomy for the direction 
and method of study. 

In the current r aper-oriented system, lessons 
are structured • ound questions that are posed 
to students. Solutions to problems are 
provided, but students cannot ask the paper 
lessons questions or for further elaborations. 
With electronic lessons, they can. We must 
make sure that we understand the line that 
separates presenting expository material, 
asking practice questions and providing 
answers to practice, and becoming teacher- 
centered. Assuming this line can be and is 
clearly drawn, then a great new domain of 
opportunity opens before us. 

We can begin to think of groupware that 
presents expository material and then provides 
practice for the concepts and relationships of 
which this material is composed, all the while 
using the best information available. 
Furthermore, as with earlier individualized 
systems, the kind of practice presented to the 
students can be continually updated to present 
the most challenging problems to the students 
without allowing them to become discouraged 
or stymied by the material being studied. 

The Role of Recall In Groupware. Another very 
significant issue to be dealt with when 
thinking about the role of computers and 
expert systems in team study is the role of 
recall. I advocate down playing recall ability as 
an instructional objective. That is, the t?am 
study method excels in the achievement of skill 
or performance objectives. When students 
learn to solve problems, the examinations or 
quizzes they should be given are application^. 
Recall is what information- processing 
software does much better than humans. I 
think groupware should emphasize problem- 
solving skills and not recall. The objectives for 
students of the new millennium should be on 
processing information not storing it. 

The recent major review of the research on 
human memory (Schwartz and Relsberg 1991) 
suggests that we do not know all that much 
about it. We know that we remember what we 
use frequently. Everything else can be retrieved 
mechanically. Thus, is it. not reasonable to 
expect students to recall how to use the 



information retrieval methods and let other 
recall objectives slide by the wayside? 

Given the ubiquitous calculator and micro- 
computer, why do wt even bother with 
arithmetic? There was a time when square 
roots had to be extracted long hand. Have we 
not come to that same place for fractions and 
the division algorithm? It is past time for us to 
be considering the nature of problem-solving 
and leaving behind the algorithms done so well 
by the machines. The mind is born free and 
everywhere it is chained to algorithms. 

Conclusion. I think it is obvious that as we 
further refine the theory of the qualitative 
structure of classrooms, we will further clarify 
the defining characteristics of groupware. 
Moreover, it is already clear that there are two 
distinct concepts of groupware. Moreover, it is 
important to keep in mind that under the team- 
study approach the role of the computer cannot 
serve an executive or directional function on a 
levei that is equal to or superior to that enjoyed 
by the students. The team-study classrooms 
place student autonomy has the highest level of 
directional authority within the pre- 
established structure of the lessons. 

Dewey's (1916) famous tome on education 
argued that the reconstruction of experience 
that adds to the meaning of experience and that 
Increases the ability to direct the course of 
subsequent experience is the central me ng 
of 'education'. Moreover, it is education L Jiis 
sense that Is the highest human value. As he 
put it, Education has no end beyond itself." 

We can extend this point for our time by 
observing that anything that enhances t>e 
reconstruction of experience is of instruments 
value. Some writers refer to computers as 
"power tools for the mind" (Feigenbaum, et. al. 
i98£., With computers we can do mind things 
that we could never do otherwise. Computers 
are power tools for the reconstruction of 
experience, and they can support our 
reconstructions of experience in ways never 
before possible. 
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METHOD AND FREQUENCY FOR USE OF A COMPUTER-BASED 
DRILL IN THE LEARNING OF VERBAL INFORMATION 
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Abstract 

The purpose of this study was to determine an 
effective schedule for the frequency and 
spacing of drill and practice in the learning of 
verbal information, and to investigate the 
number of times to review a missed item during 
a drill. 

The study used a computer-based drill which 
incorporated in its structure the findings of 
past research. It involved 117 junior high and 
high school French I students who drilled 20 
French words. Half practiced twice, and the 
other half practiced three times. One third 
reviewed missed items once, twice, or three 
times respectively. After two weeks of 
scheduled practices, a posttest was given, 
followed by a delayed posttest one week later. 

Analysis showed that both treatments resulted 
In a significant difference between the delayed 
posttest means. That is. those who practiced 
three times scored significantly higher than 
those who practiced twice. Also there was a 
significant difference between repeating a 
missed item once and repeating it three times, 
but not between 1 and 2 times or 2 and 3 times. 
The verbal ability of the students, as measured 
by the Iowa Test of Basic Skills, did not have a 
significant effect on their success with the 
drill. 

This study, conducted In a school setting, offers 
suggestions for ways to structure our drill and 
practice procedures In that setting so that their 
impact on learning is maximized. 

Introduction 

Drill and practice has gotten some negative 
publicity in recent years. One reason is that as 
we use and study the use of modern technology 
in education we are becoming more aware of 
the various levels of knowledge. In comparison 
to higher level skills such as problem solving, 
drill and practice can seem meaningless and 
unimportant. Drill and practice is considered 
by some a poor use of the computer, both 
because most drills do not use many of the 
capabilities of the machine and /or because 
many computer drill programs are of poor 
quality (Fuson, 1985; Salisbury, 1985; Yates, 
1983). In spite of this fact, research in modern 



cognitive theory suggests the importance of 
drill and practice in the learning process 
(Merrill. 1984). 

Since the microcomputer entered .ae 
classroom, many computer-based drills have 
been developed for use in schools. Such drills 
can be effective research tools because it is 
possible to control the structure of the drill as 
well as its use. After the Introductory section 
of the drill. Jn which instructions and 
explanations are given, there is usually a cycle 
of steps which is repeated for each item: 
selection of the item, display of the item, the 
response of the learner, a Judgement of the 
response, and feedback to the learner. Each of 
these steps can be designed by the researcher. 
Then, as the drill is used, the length, spacing, 
timing and number of drill sessions can be 
controlled (Rysavy, 1991). 

Research has studied some of the aspects of 
drills In order to determine which 
qualities/methods are most effective. Using 
the results of this research, a study was 
conducted to determine an effective schedule 
for the frequency and spacing of drill and 
practice in the acquisition of declarative 
knowledge. It also investigated the number of 
times to review a missed item during a drill. Its 
purpose was to further refine what we already 
know about drill procedures so that we can 
maximize learning and minimize the number 
of practices needed for learning. 

Background for Study 

At the center of this study are the concepts of (1) 
verbal information and (2) drill and practice. 
The term "declarative knowledge" is usually 
considered to mean the same as "verbal 
information." but "verbal information" is more 
commonly used in the literature. The terms 
will be used interchangeably in this paper. 
After a short discussion of these concepts, 
previous research on various aspects of drill 
and practice will be summarized. 

Verbal Information 

We learn a great deal of verbal information 
throughout our lives, from the names of people, 
objects and concepts, to more highly organized 
information, such as events of history or 
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achievements of lndustiy and science. Areas of 
specialty each involve a particular vocabulary; 
e.g. a botanist needs to know names and 
information about plants, and a chemist needs 
to know the chemical symbols. Rabinowltz & 
Glazer (1986) stated that "an expert in a given 
domain is thought to have a greater amount of 
declarative knowledge about that domain than 
does a novice" (p. 78). 

At times, verbal information is critically 
important as a basis for other learning (Dick, 
1985). Examples include knowing events of 
U.S. history and government in order to 
analyze the influence of political parties in the 
United States, and learning the vocabulary in a 
foreign language as a basis for a very complex 
set of communication skills. What enables a 
person to utilize such information as a base for 
the more complex tasks is an automaticity to 
Its use (Dick, 1985; Gagn*, 1977; Merrill. 1984). 
To facilitate acquisition of automaticity, the 
development of effective methods for learning 
and rehearsing that declarative knowledge is 
important. 

Drill and Practice 

Drill is defined to be "the process of training or 
teaching by the continued repetition of an 
exercise," while practice is "to do repeatedly to 
order to learn or become proficient" (Guralnik, 
1986. pp. 427. 1117). While both involve doing 
something over and over, drill is considered 
repetition of a skill to order to learn it, while 
practice Is repeating it once it has been learned 
in order to become more proficient to it. This 
distinction Is not important for this paper. 

Drill and practice has traditionally been used 
for the learning of paired associates. During 
the last decade, research to cognitive learning 
theory has suggested that drill and practice 
may be more important than many realize, and 
can have a significant role to learning 
(Salisbury, 1984). Practice of verbal 
information, which provides review of the 
material, can result to additional and more 
elaborate encoding (Gagn\ 1977). 

Previous Research 

Previous research suggests procedures and 
methods for the development of computer drill 
programs. This research studied short-term 
memory, review of missed items, feedback, and 
spacing and length of practice sessions. A 
summary of these topics is followed by 
considerations regarding the learner. 

Short-term memory. 

According to modern views of learning, short- 



term memory is a kind of memory which stores 
limited amounts of information for a brief 
time interval (Gagn», 1987). According to this 
view, information is encoded and enters long- 
term memory once it is processed Into a 
meaningful context. Research has suggested 
that short-term memory is limited by the 
number of items it can handle at one time 
(Miller, 1956). This limitation suggests that for 
drills, items should be grouped, when this Is 
appropriate. The number of items being drilled 
at one time can also Tx kept small by dividing 
the items into a working pool and a review pool 
(Salisbury, 1988). A limited number of words 
are used from the working pool at a time, and 
are dropped from that set only when they are 
learned. 

Review of misted Items. 

When drilling verbal information, the learner 
can make errors. Those items should then be 
reviewed until they are learned correctly. 
Research has studied both how often and how 
soon a missed item should be reintroduced. 

Regarding how soon a missed item should be 
reintroduced, Jacoby and Craik (1979) found 
that immediate repetition of a missed item 
results in no further useful encoding. 
Goldenberg (1987) found the best position to 
reintroduce a missed item before it is lost from 
short-term memory is after 2 intervening 
items. The next best position is after 3 
intervening items. As to how often a missed 
item should be reintroduced, it has been 
suggested that missed items be reviewed up to 3 
times (Salisbury. 1985). Siegel & Mlsselt (1984) 
concluded that increasing ratio review 
techniques (e.g. after 2 intervening items, again 
after 3 more Intervening items, and once more 
after 5 more intervening items) are more 
effective than 1 later or no review at all. 

Feedback. 

For purposes of drill of verbal information, 
immediate feedback Is most desirable (Cohen, 
1985). It can be scheduled after both correct 
and incorrect responses, but it is more 
Important after errors (Kulhavy, 1977). 

Research has shown that feedback with 
discrimination training is effective (Siegel, 
1984). A "discrimination" error occurs when 
the learner gives a response which is correct for 
another item to the drill list. The feedback 
given for this type of error includes informing 
the learner both of the correct response as well 
as of the item which matches the answer given. 

Spacing and length of practice sessions. 
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Researchers have asked the question as to how 
much and how often a learner needs to practice. 
Much evidence has suggested that short, spaced 
practice periods yield better results than long 
concentrated practice periods (Anderson, 
1980). Reynolds and Glaser (1964), for example, 
found that the practice of new material intact 
(massed) was relatively ineffective. Gay (1973) 
found that placement of a practice near the 
time of learning (early) and another near the 
retention measure (late) are most effective. 

The learner. 

For the learning of verbal Information, as for 
any type of learning, the learner must be 
considered in the choice of instructional 
materials and methods. These considerations 
include the learners' preparation for the 
material to be learned (aptitude, motivation, 
prior knowledge, cognitive structure), and the 
method used (Ausubel, 1963, 1968, 1969; 
Bloom, 1976; Snow, 1980; Tobias, 1987). The 
learners should be informed about how the new 
material will fit into what they already know. 
The objectives should be clearly stated, 
specifying the desired learning outcome (Gagn*. 
1977). If the student is involved in other 
learning activities at the same time, the effect 
of these activities should be considered 
(McGeoch, 1932). 

Implication* of the literature and development 
of a pilot study. 

The issues discussed above (learning of verbal 
Information, need for drill and practice, and 
short-term memory) provide the rationale for 
the material of this study: use of a drill for the 
learning of verbal Information. The findings 
regarding review of missed items, feedback, 
and spacing and length of practice sessions, 
were used to create and Implement a computer- 
based drill which would be as effective as 
possible. A pilot study, using that drill, was 
conducted to generate additional data 
regarding the question of how much practice 
was needed to effectively learn verbal 
information. That data was used to formulate 
the hypothesis for the main study. More 
complete details on both the pilot study and 
main study are found in Rysavy (1991). 

Pilot Study 

Subjects for the pilot study consisted of 28 male 
and 34 female 4th, 5th and 6th graders in an 
urban public elementary school. The students 
were given the task of learning the capitals of 
the 50 states. Using the suggestion of Tobias 
(1981) that research might be begun by 
observing what students of varying individual 
difference characteristics actually do and what 



instructional options they select, each student 
was given the task of learning the material, 
and made his/her own decision as to when and 
how long to practice. The drill was designed so 
that the student could stop at any time and then 
continue through the drill at the next practice 
session. 

Each student had an individual disk which 
contained the drill program and files which 
stored details and data of each practice session. 
At the beginning of the study, students were 
given a written pretest. It was explained to 
them that they were going to help researchers 
better understand how students learn. They 
were urged to , take seriously their task of 
learning the states and capitals within the 2- 
week period and to decide how much practice 
they needed to accomplish that task. At the end 
of two weeks a posttest was given. 

In order to determine which variables which 
most affected the learning of the verbal 
material, as many aspects as could be 
quantified were studied. These included pretest 
and posttest scores, number of times the 
student practiced, when the student practiced, 
number of times through the entire drill, and 
length of practice sessions. 

The percentage of items correct on the posttest 
after the various number of times of practice 
was calculated. Percentages were calculated on 
all items, as well as on those which were not 
known at the time of the pretest. These results 
are shown in Figure 1. 

PERCENT OF ITEMS CORRECT AFTER PRACTICE 
Percent Cornet 

120 1 ■■ , 




O1235 



Number of Practices 
■I all Ittmt B not known tt prtttst 

Figure 1 Percent of Times Correct After 
Practice 

These results suggest that when an optimal 
drill format is used (discrimination training, 
increased ratio review of 3 positions for missed 
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items) with placement of reviews early and late 
(as described above), then two times through 
the drill are sufficient for maximal learning of 
the verbal information being practiced. 
Regarding the number of times an item is 
reviewed once it is missed, the study showed 
that a review of 3 times is effective. Details of 
the other variables considered are found in 
Rysavy (1991). 

Main Study 

Hypothesis 

Using the results given above, the study was 
based on a hypothesis as well as a question. 
They are as follows: 

(a) Hypothesis: two times (placed early and 
late between initial learning and the 
retention measure) through a drill which 
utilizes discrimination training and the 
increased ratio review already described, 
is as effective as three times through the 
drill. 

(b) Question: regarding the number of times a 
missed item is reviewed, are 2 times or 1 
time as effective as 3 times? 

Study Procedures 

One hundred seventeen Junior high and high 
school French I students were divided into six 
groups stratified by grade level. Their teachers 
were consulted to determine a list of French 
words which the classes had not yet studied but 
which would build on their course work. A 
computer-based drill containing those 20 
French words and their English equivalents 
was created. Half of the subjects practiced 
twice, and the other half practiced three times. 
One third reviewed missed items once, twice, or 
three times respectively. The drill used 
discrimination training for all subjects. After 
two weeks of scheduled practices, a posttest was 
given, followed by a delayed posttest one week 
later. 

Study Results 

Pretest scores had a mean of .98 out of a 
maximum of 20. The mean of the posttest was 
16.2, and that of the delayed posttest was 16.1. 
Some of the students were not able to practice 
all words during one or more practice sessions. 
These 40 words, along with the 108 words 
known before the study began, were removed, 
and the posttest scores were re-calculated as 
percent of unknown words learned. The 
resulting mean of the first posttest was .81, and 
of the delayed posttest was .80. The correlation 
coefficient between the first and delayed 
posttests (Prunkl. 1979) was 91.2%. Details of 
posttest scores for the treatment groups are 



given in Table 1. 
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Figures 2 and 3 show the comparison of two 
and three practices, and the comparisons 
between one, two, and three repetitions of 
missed items, respectively. 

Analysis showed that both treatments resulted 
in a significant difference between the delayed 
posttest means (p. = .015 for number of 
practices, and fl = .003 for number of times a 
missed item is repeated.) That is, those who 
practiced three times scored significantly 
higher than those who practiced twice. Also, 
using Tukey's Studentlzed Range Test, a 
significant difference at the .05 level was found 
between repeating a missed Item once and 
repeating it three times, but not between 1 and 2 
times or 2 and 3 times. The verbal ability of the 
students, as measured by the Iowa Test of Basic 
Skills, did not have a significant effect on 
their success with the drill. 

These results suggested the following regarding 
the hypothesis and question proposed by this 
study: 

Hypothesis. 

The hypothesis, based on the pilot study, was 
that there would be no significant difference in 
posttest scorr-; between those who practiced 
twice and those who practiced three times. The 
main study showed that there was a significant 
difference. At the same time, subjects whose 
drill procedure was identical to the pilot study 
(i.e. reviewing missed Items three times) 
differed by only 3 percentage points »n mean 
posttest scores, compared to 12 and 14 
percentage points difference between the groups 
reviewing missed Items 1 and 2 times 
respectively. This suggests that reviewing 
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missed items 3 times may reduce the number of 
practices needed. 

Question. 

This study asked the question of whether 
reviewing a missed item 1 or 2 times would be 
as effective as reviewing it 3 times. The study 
data showed a significant difference between 1 
and 3 times, but not between 1 and 2 times or 2 
and 3 times. This suggests that once is not an 
effective number of times to review a missed 
item. The effectiveness of 2 reviews is not 
clear. 
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Figure 2 Comparison of Two and Three 
Practices 

Other Results 

The result of student choiCwS during the pilot 
study concerning frequency and length of 
practice sessions supported the findings of Gay 
(1973) regarding the time of practice. Gay found 
that an early review and a late review are better 
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than two early or two late reviews. In the pilot 
study, the average amount learned by students 
who practiced both weeks (both early and late) 
tM=.70, 14=22) was twice the average amount 
learned by those who practiced only early or 
late (M=.35, N=27). The effectiveness of this 
schedule was further tested In the main study 
through the administration of a delayed 
posttest. This posttest showed that student 
retention did not decrease significantly after a 
week with no further practice. 

Recommendation! "or future research 

This study, along . ith those on which it is 
based, shows that when meaningful verbal 
Information is to be learned, effective methods 
and timing of drill and practice can maximize 
learning and minimize the number of practices 
needed. The drill process, however, can be 
affected by other facets of learning both 
Internal and external to the learner, as well as 
by the Instructional process surrounding the 
drill. Several of these issues suggest questions 
for future research. Some of them are as 
follows (Rysavy, 1991): 

(1) How do the Instructional procedures used 
before and after the use of a drill affect the 
amount of practice needed? In this study, 
would the students have needed less 
practice If they had been guided through an 
initial learning of the words before the drill 
sessions? 

(2) Are there ways in which a drill procedure 
can help learners group or chunk verbal 
material as they drill it? Would the 
grouping of similar items, or items which 
may be confused, help the learner to 
differ tlate between Jiem and thus learn 
them niore quickly? 

(3) Given a period longer than two weeks, will 
the number of times through the drill 
needed to learn the verbal information 
increase? If the period between initial 
learning and the retention measure was one 
or two months, for example, would 
additional practices be needed? 

(4) Given a longer or shorter drill, will the 
amount of practice needed increase or 
decrease? Length of drill, which depends 
both on the number of items and on the 
number of mistakes made, can affect the 
attention. Interest and retention levels of 
the student. 
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Figure 3 Comparison of One, Two and 
Three Repeats of Missed items 



(5) How is the amount of practice needed for 
effective learning affected by the other 
learning activities In which the learner Is 
involved during the same period? Do 
students In a school setting need more 
practice than those learning verbal 
Information outside the school setting 
(when they are not Involved In other 
learning activities)? 

Conclusion 

When considering use of drill and practice, the 
first question Is always whether It Is 
appropriate for the material being studied. 
This question Is best answered by experts to 
each field. (Note: Use of drill and practice for 
the material of this study Is discussed to 
Fysavy (1991).) Given the appropriateness of 
drill and practice, this study, conducted to a 
school setting, offers suggestions for ways to 
structure drill and practice procedures to that 
setting so that their Impact on learning is 
maximized. Some of these implications, along 
with others which are suggested by previous 
research, include the following: 

(1) Feedback to student response should be 
chosen which is most appropriate to the 
material being learned. For matched pairs, 
feedback with discrimination training is 
effective. 

(2) Shorter spaced drills are more effective 
than massed drills. 

(3) Computer-based drills can keep track of 
student responses, thus repeating missed 
items at appropriate intervals and also 
allowing the student to stop at any time and 
finish the drill at another sitting. 

(4) Items which are answered incorrectly 
should be repeated two or three times at 
increasing intervals. 

(5) When possible, similar or related items can 
be grouped to order to help the learner 
distinguish between the items and learn 
them more effectively. 

(6) Two times through the drill placed early 
and late between the time of learning and 
the retention measure are most effective. 

Additional studies need to be conducted to 
learn more about ways to structure drill and 
practice so that their Impact on learning Is 
maximized. Some of these results can also help 
to Improve our instructional methods for other 
types of learning situations. 
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Introduction 

Today's state-of-the-art computer assisted 
Instruction (CAI) systems have their roots in 
the IBM- 1500 system, the PLATO system at the 
University of Illinois, the PDP-10 system at 
Stanford University, and other institutions In 
the 1960s. The effectiveness of CAI has been 
demonstrated and reported by numerous 
researchers during the last twenty years (cf, 
Kulik & Kulik, 1987). Although the high cost of 
CAI delivery systems developed in the early 
years restricted the extent of their 
dissemination, the development of 
inexpensive, powerful microcomputers in 
recent years has significantly reduced the cost 
of hardware required to support CAI. 

There is no longer much debate regarding the 
inevitability of CAI being used for education 
and training throughout the industrialized 
world. As Industrialized nations move rapidly 
towards computer-Intensive classrooms, 
schools, and school systems, developing 
nations are beginning to feel the pressure of 
being left behind. Even if computer education 
in schools lives up to only a fraction of the 
potential claimed by CAI proponents, it will 
certainly widen the gap even more between the 
developed and developing countries. Therefore, 
it is necessary for educators of the third world 
to look into more feasible ways of introducing 
CAI into their school systems. 

The integration of CAI into the schools of 
developing countries Is not as simple as Just 
accepting the inevitability of an increasingly 
computerized society. Marshall (1984) 
identified critical areas of discussion for 
decision makers concerned with the 
introduction of computers into the third world 
educational systems. Thes2 Include financial 
considerations, language and cultural 
differences, preparation and training of 
trainers, hardware selection, software 
availability, and courseware compatibility 
with culturally and politically unique 
educational philosophies and settings. Jones, 
Harmon. & Reeves (1990) developed a ten 
component model for analysis of the 
feasibility of computer-based instruction In 
developing countries, including strategies for 



dealing with cultural sensitivity, technological 
appropriateness, and conviviality. However, 
each of these models is focused on a country- 
level analysis. They do not provide detailed 
guidance for actually implementing CAI in 
developing countries. The purpose of this 
paper, on the other hand, is to recommend a 
model for effective development and 
implementation of CAI in developing 
countries. A major assumption of this model is 
that software rather than hardware presents 
the major challenge in this effort. 

Educational Computing Implementation 
Problems 

The success of educational computing in any 
country is very much dependent on the 
availability of high quality software. 
Although there are numerous computers in 
schools in developing countries, good 
educational software relevant to local 
educational curriculum and setting is virtually 
nonexistent. One major reason for this 
scarcity is the lack of financial incentives for 
educational software development. There Is 
little financial reward to be gained from 
developing such software since the market Is 
small and most schools lack money to Invest 
in software purchase. Another reason is the 
lack of computer programmers with relevant 
educational experience and expertise. As a 
result, the small amount of available 
commercial software lacks the pedagogical 
considerations to make it instructional^ 
sound for classroom implementation. 

Developing countries have Jumped on the 
bandwagon of Implementing some form of 
computer education program to ensure that the 
technological and educational gaps between 
them and the developed, nations do not 
increase. In the excitement of being in-step 
with the developed nations in computer use in 
schools, these developing nations have 
repeated similar mistakes encountered by the 
developed nations during the infancy of their 
use of computers In education (Friend, 1985). 
Some of these countries have Invested large 
sums of money In equipment that is now 
obsolete. The major challenge faced by nearly 
all the developing nations, however, is the 
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availability of software that is compatible and 
relevant to their unique educational and 
environmental settings. 

The Lack of Educationally Sound Software In 
Developing Countries 

Nearly every developing country has decided to 
spend a major portion of their education budget 
towards implementing computer use in 
schools. These countries are now finding out 
that the availability of the appropriate 
hardware has automatically made their 
educational computing programs effective. 
Most of them are now facing the problem of 
obtaining software that Is both relevant and 
culturally sound for their specific educational 
contexts. 

In Taiwan, R.O.C., the Ministry of Education 
decided to launch a major effort in 1986 to 
develop CAI courseware for a wide range of 
subjects. This project produced a total of 417 
courseware units for vocational high schools 
within four years. This experiment was 
cancelled after realizing the dearth of 
appropriate software available for use in 
schools (Hong, 1989). Many of the CAI lessons 
developed in Taiwan were unsatisfactory 
because they were Just •page-turners" (Alessi & 
Shlh, 1989). Computer educators in Taiwan 
have come to realize that good CAI does not 
come simply from placing computer hardware 
In schools. The presence of computer hardware 
provides a necessary, but insufficient basis for 
successful CAI. Teacher training and good 
software development are also necessary. 

The Ministry of Education in Bahrain decided 
to carry out a pilot project in 1985 to introduce 
computer education into secondary schools, 
and they were faced with four major problem 
areas: hardware, software, Arabic language 
interface, and teacher .raining. The problems 
of hardware uniformity and compatibility as 
well as teacher training were resolved 
somewhat easily (Chaudhry & Fakhro, 1986). 
In addition, the Arabic Interface problem was 
remedied when a company designed a computer 
system supporting Arabic characters. The 
most pressing problem encountered was the 
software problem. The Bahrain authorities 
decided to develop their own national software, 
but they have encountered many problems In 
designing software that is instructional^ 
sound and relevant. 

In the mid-1980s, the People's Republic of 
China conducted a study to find out the 
potential of implementing CAI in its 
educational system. Among the primary 



recommendations forwarded was that China 
should develop its own CAI material to ensure 
that it would be relevant to the Chinese school 
system. In their examination of existing 
software, they found an almost total lack of 
programs that could be easily used within their 
unique educational settings (Wu & Yu, 1987). 

Although the Soviet Union is not a part of the 
Third World, the USSR is increasingly 
recognized as sharing many of the problems as 
those nations traditionally labeled as 
developing countries. Hoot (1987) estimated 
that even if the Soviet Union installed 
computers in every school, the computers 
would probably gather dust because of the lack 
of software. Major contributors to the software 
dilemma In the USSR are that programming 
languages are mostly in English and their 
uncompetitive economic system does not 
motivate programmers to invest time and 
effort in producing educational software. 

Australia, Japan, India, Philippines, Sri 
Lanka, Thailand and Malaysia have all aired 
similar views concerning the major problems 
encountered In their attempts at Implementing 
computer education in their respective 
countries. These views were voiced in the Third 
As&n Seminar on Educational Technology 
held at the Tokyo Gakugei University, Japan in 
1984 (APEID, 1985). Australia and Japan, 
however, have moved ahead towards the 
solution of the software problem with the 
abundance of expertise available in their 
countries. The rest of the developing world Is 
still struggling to find a solution that is both 
feasible and economical. 

Software Development m Developed Countries 

Considerable effort has been made by many 
countries to produce educational software of 
better quality. Different strategies have been 
used to support the software development and 
production process, examples of which are 
individual, team, and industrial or 
commercial approaches (Moonen, 1989). In 
general, the quality of software is improving 
and the quantity of educational software is 
increasing. However, this is mainly true for 
the already developed nations. The 
development and production costs of these 
approaches remain too high for many 
developing countries. Nonetheless, some 
lessons can be derived from the c 1 liferent 
strategies for software development employed 
in industrialized countries. 

Reporting on the development of educational 
software in The Netherlands, Moonen and 
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Schoenmaker (1986) concluded that software 
development approaches which have been 
supported by an individual developer usually 
have to Incorporate more people after a while, 
or fail as a result of the frustrations of 
Insufficient recognition, insufficient 
compensation for Investments, or insufficient 
perspectives with regard to the work carried 
out. Although the individual approach to 
courseware development might seem very 
appealing to developing countries, the lesson 
from Moonen and Schoenmakei's research 
should be heeded. 

Another approach that is frequently used is the 
team approach. Developing computer-assisted 
learning materials requires a philosophy and 
strategy which places the technology and the 
curriculum on equal footing. This has long 
been recognized by serious developers of 
educational software. Watson (1933) 
maintained that CAI development should be a 
team activity, depending upon the interplay of 
different expertise representing hardware, 
software, curriculum and classroom. The 
Computers in the Curriculum (CIC) project, 
based at King's College (Chelsea), London, Great 
Britain, has developed CA f for pupils in 
secondary schools (11-18 years of age) since 
1973. 

The CIC project has published over 150 CAI 
units in various subject areas. Its development 
model reflects expertise in both the technology 
and the curriculum (Watson, 1987). According 
to Watson (1989), the success of the model 
depends upon a large development team in 
which several groups play different roles. 
Creative and energetic teachers provide 
practical understanding of their syllabus and 
situations; curriculum developers ensure that 
appropriate and current areas are tackled; 
professional programmers code good sound 
software; and CAI developers (or instructional 
designers) specialize in screen design, flow, and 
user interface aspects of the software. This 
combination of interlocking expertise and 
talents has resulted in effective and sound 
educational software for use in British schools. 

The Netherlands used a similar approach 
where different teams are responsible for the 
pedagogical design, the system design, and the 
actual coding. Different approaches using the 
team concept have been tried out, but no 
satisfactory solution has been found for a 
completely effective development system. The 
mam problem appeared to be communication 
between the developers and the target group 
(Moonen, 1989). Belgium. Denmark, France, 



Germany, Luxembourg and Spain have 
developed national software development 
bv . r, .e» which mainly use different versions of 
the team approach (Plomp. Van Deursen & 
Moonen, 1987). 

Schools in the U.S.A. on the other hand, 
usually obtain educational software by 
purchasing individual software packages from 
commercial publishers. More than 10,000 
software products Intended for instructional or 
educational use with computers in schools and 
home can be found on the U.S. market (Hunter, 
1989). Commercial publishers in the U.S. have 
incorporated the team approach and combined 
it with their commercial/ industrial approach 
to churn out educational software. Many of 
these software packages have been sold to 
developing nations; however, for the most part, 
they were suited only to the U.S. curriculum, 
educational methods, and culture. 

A major impetus for the team approach was the 
difficulty educators faced in developing 
software on their own due to complexities of 
coding or programming. Until recently, there 
was a lack of simple-to-leam authoring tools 
that were powerful enough to create the 
sophisticated instructional transactions that 
needed to be performed. All the design ideas 
had to be translated using high-level 
programming languages which educators found 
difficult to master. Many of those Involved In 
team approaches to software development 
since the early 1980's have agreed that the 
advent of powerful authoring tools has changed 
their views about future approaches to 
educational software development 
(Moonen, 1989; Watson, 1989; Hunter, 1989; 
Plomp etal., 1987). 

McLean (1989), in a paper about megatrends of 
educational development, suggested some sort 
of a return to the early approaches in software 
development, specifically the development of 
software by practicing educators themselves 
rather than commercial development. Watson 
(1989) noted that the rapid growth of software 
authoring tools Is going to have a great impact 
on the future of software development in the 
United Kingdom. Although the proliferation of 
authoring systems requires modifications in 
the development process, it does not change the 
underlying team-based philosophy of how 
software should be developed. Instead of 
having four separate teams doing work in their 
own areas of expertise, the presence of new 
simple, powerful authoring tools has shifted 
the focus of development to a smaller 
centralized team consisting of a 
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designer/author, subject matter expert, and 
programmer. This has important implications 
for developing nations lacking sufficient 
financial and expertise resources for 
commercial development of educational 
software. 

Educators as Courseware Developers 

In 1985, over sixty different authoring systems 
were reported (Locatis & Carr, 1985). This 
marks a growing trend to make authoring tools 
available to those who generally lack 
programming and courseware development 
experience. The availability and ease of use of 
authoring software such as HyperCard, 
Authorware Professional™ , Icon Author , 
Scriptwriter, Linkway, Tutor Tech and many 
others have increased the interest and 
involvement of educatois in educational 
courseware development (Hebenstreit, 1989). 
One factor inhibiting software development by 
teachers previously was the conflict between 
the need for attractive screen displays and easy 
navigation techniques in software to be used by 
students and the time i training necessary to 
develop those aspects. Most of the authoring 
tools mentioned above have made it feasible 
for teachers to design screens that are both 
attractive and functional. 

In the early 1980's, there was much discussion 
as to whether teachers should be Involved In 
developing educational computer courseware. 
The general consensus were that teachers 
should not generally be Involved in courseware 
development since most teachers developed 
software that only met their own specific local 
needs and did not contribute to the overall 
Improvement of education. Errors in teacher- 
developed courseware occurred in both the 
instructional design and programming 
characteristics (Roblyer, 1983). Opinions in 
recent years, however, have shifted due to the 
existence of more advanced authoring tools 
and the Increase in the number of teachers with 
educational computing experience. Today's 
teachers may also be more aware of the 
principles of instructional design, especially as 
applied in educational software development. 

Many administrators are again encouraging 
educators to be involved in developing 
instructional software for school use. Several 
models for the design of CAI software by 
teachers have been developed, and textbooks 
concerning the principles of CBI design for 
teachers or non-programmers have been 
published to encourage teachers in developing 
software (Tessmer & Jonassen, 1988; Bosler & 
Squires, 1989; Crossley & Green, 1985; Alessi & 
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Trollip, 1987). These models emphasize the use 
of sound principles of instructional design 
methods to be incorporated together with the 
use of the authoring tools for courseware 
development. The consensus of opinion is that 
a key factor in the successful Implementation 
of educational computing is educator 
involvement as courseware developers, 
certainly at the design stage, and even at the 
coding stage using authoring tools 
(Smith, 1987; Watson, 1989). 

The existence of powerful authoring software 
has made training of educators as software 
developers more feasible (Bosler & Squires, 
1989). This may benefit developing countries 
since hiring of programmers for software 
development is both very expensive and 
difficult due to the lack of programmers in 
these countries. Many developing countries 
have used educators as software developers by 
training them to program using high-level 
programming languages to overcome the 
shortage of programmers. However, the 
experience of Taiwanese educators indicates 
that programming courses or workshops are 
not sufficient for courseware development 
(Alessi & Shlh, 1989). 

Like Taiwan, many developing countries have 
implemented computer literacy training to 
encourage teachers to develop software. 
Similar problems appeared in these countries 
and very little educational software has been 
produced. Those programs produced were of 
low quality, especially In instructional design 
considerations (APEID, 1985). Trying to train 
teachers to program using high-level 
programming languages in a short duration of 
time, usually 1 week - 3 months, is Ill-advised. 
Such an approach neither turns teachers Into 
competent programmers nor helps them to be 
designers of good software. 

Other factors that influence the quality of 
software developed by teachers are inadequate 
monetary support and the lack of a coordinated 
systematic plan for software development. Our 
review of the literature concerning 
methodology of software development in both 
developed and developing nations has revealed 
several important points that should be 
considered for the implementation of 
educational software development projects in 
developing countries: 

1. a team approach is necessary to ensure a 
well-designed courseware products; 

2. the use of teachers as designers and 
deve?opers should be pursued due to financial 
and expertise constraints; 




3 teachere must be trained In the principles of 
Instructional design and implementation of 
Instructional innovations; 

4. slmple-to-use but powerful authoring tools or 
authoring systems should be used; 

5. a coordinated approach with a systematic 
monitoring and evaluation should be 
established at the national level; and 

6. more financial support and involvement 
should be present from the educational 
agencies responsible for educational 
computing. 

Current Software Development Models 

A software development model that could be 
used by developing countries can be derived by 
combining the team approach carried out by 
Computers In the Curriculum (CIC) project of 
the U.K. and practices carried out in The 
Netherlands, albeit on a smaller scale (Watson. 
1983; Moonen, 1986). The-CIC model utilizes a 
large team reflecting a combination of 
expertise in hardware, software, curriculum, 
and pedagogy. The teams consisted of 1) 
classroom teachers In a variety of subjects; 2) 
curriculum developers; 3) professional 
programmers; 4) system analysts; and 5) CAL 
developers. The team was subdivided into a 
writing team and a central team. The writing 
team consisted of classroom teachers, 
curriculum developers, group coordinators, 
and programmers. This team was responsible 
for the main bulk of the development work. 
The central team consisted of a chief 
programmer, a systems analyst, a software 
support manager. CAL developers and 
programmers. Although a significant 
proportion of the work was carrltd out by the 
writing team, the central team tock an active 
developmental role as well as a managerial one 
from its central base In Chelsea, London 
(Watson, 1987). 

The Netherlands approach was similar to the 
CIC approach. The Dutch government created 
three development centers, each one specially 
aimed at one particular sector of education. 
These development centers were mainly 
Involved in developing prototypes of software 
for commercialization. They also supported an 
agreed-upon methodological approach for 
software development by smaller groups in 
their vicinity. The development groups focused 
their activities mainly towards the coding 
aspects of the development process whilst the 
design process was the responsibility of the 
development centers. 

A Modified Courseware Development Model 

The rest of this paper outlines a system or 



methodology for developing instructlonally 
sound educational courseware that Is relevant 
to a developing country's unique educational 
setting. This is the beginning of a model for 
courseware development that is both 
economical and effective for small developing 
nations with little financial and expertise 
resources. It is hoped that this generic model 
will provide the guidance necessary for the 
successful diffusion and implementation of 
CAI in the educational systems of developing 
countries. 

The Team Approach. 

The basis of the modified software 
development model lies In a team approach 
that reflects a combination of expertise in 
Instructional design, software, curriculum, and 
schools. Each team will consists of the 
following: 

a) an instructional designer (as a coordinator 
and resource person); 

b) a teacher (as a designer- developer); 

c) a subject matter expert (as a curriculum 
resource person); and 

d) an authoring software expert (as a software 
resource person). 

It Is important to realize that the team size is 
small due to the considerations of the financial 
and expertise constraints faced by developing 
countries. However, several small teams such 
as these spread out throughout the country 
would help Increase the rate of implementation 
of the software developed. 

a) The instructional designer as a coordinator 
and resource person: The Inclusion of the 
Instructional designer is to ensure that the 
software development follows the basic 
guidelines of sound instructional design, 
content integrity, and relevance to the local 
curriculum. The Instructional designer should 
be a person with expertise In instructional 
design and computer-based Instruction, 
preferably with experience In using the various 
authoring tools used for the software 
development. He/she will also train the other 
team members about the basic principles of 
instructional design and computer-based 
instruction. 

The Instructional designer should also act as a 
coordinator overseeing the management and 
administration aspects of the development 
team. He/she should establish a close 
relationship with the team members and act as 
both a catalyst for ideas and as a bridge 
between the two main types of work, i.e., the 
authoring and the educational. Establishing 
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connections to other resources, including 
hardware and software expertise, will be a very 
Important aspect of this work. He/she will 
usually be In charge of not one but several 
development teams Involved in the 
development of software in similar areas of the 
curriculum. Another critical role for the 
instructional designer will be guiding 
formative and summatlve evaluation. 

b) The teacher as designer/ developer: Creative 
and energetic teachers are essential for the 
development of good software. A teacher of a 
given discipline can provide a practical 
understanding of his or her subject's structure 
of knowledge and methods of instruction. For 
example, a geography teacher is the best 
qualified developer of courseware for 
geography lessons since he/she instinctively 
knows a variety of ways a topic may be taught 
in the classroom and the areas of learning 
difficulty. This knowledge and the 
accompanying pedagogic skills can be tapped 
and transformed Into sound software 
principles. 

The teachers must be trained in the basic 
principles of Instructional design and Its 
application and relevance to the process of 
educational computer courseware 
development. In addition, they must be 
rigorously trained In the use of a type of 
authoring language/tool for software 
development for a duration of not less than 
three months. The teachers should be 
Identified and selected based on their current 
knowledge, lm ,ivements, and initiatives in 
the area of computer education in their 
schools, district, state, or at the national level. 
The teachers should be handpicked through a 
process of recommendations, interviews and 
background research of opinions of their peers. 

c) The subject matter expert as a curriculum 
resource person: The software development 
process must Include the Input of the people 
with the actual knowledge of the content to be 
developed. Subject matter experts are 
knowledgeable about the curricula 
requirements of the school system. They 
should act as advisors to the teacher- 
developers to ensure the software follows the 
curriculum guidelines and educational 
philosophy of the local school system. 

The subject matter experts must also work 
closely with the instructional designer in 
planning in-service training courses for the 
implementation of the courseware In 
classrooms. They should work with the 
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teacher-developers in the testing, evaluation 
and the development of classroom-use 
strategies for the courses. These courses should 
be conducted by the subject matter experts 
assisted by the teacher-developers. 

d) The authoring software expert as software 
resource person: The in-depth training of the 
teacher-developers to use authoring tools 
should be the responsibility of the authoring 
software experts. They should work closely 
with the instructional designer/coordinator In 
planning and implementing the training 
program. They she lid also act as an advisor 
and resource person to the teacher-developers 
throughout the courseware development 
process to help fix problems encountered in 
using the authoring tools. 

During the development process, the 
courseware created by the teacher-developers 
should be periodically reviewed in terms of 
efficiency in using the authoring language/tool 
capabilities to ensure that the software created 
is systematic and using the full capabilities of 
the authoring tool. The software expert should 
periodically meet with the teacher-developers 
to monitor their progress and generate further 
Ideas and suggestions concerning the software 
being developed. 

The Development Process . There are several 
critical components that must be coordinated 
to support a systematic development process. 
These factors include: 

a) hardware considerations: Hardware must be 
chosen carefully with regards to its ease of use 
tince teachers and students in diverse 
classrooms will be using the developed 
software. The hardware chosen should not 
require teachers to undergo comprehensive 
'raining courses to be comfortable using it. 
Since one of the major constraints of 
implementing CAI In a developing country is 
limited financial resources, the shorter user 
training time would minimize the budget 
necessary for training. 

b) authoring tools /systems: Authoring 
tools/ systems selected for the teachers to 
develop their courseware should be chosen for 
their powerful capabilities In designing and 
developing of courseware as well as for their 
hardware compatibility. The systems should 
be easy to learn so that teachers can develop 
courseware with their little or no programming 
experience. MacKnight and Balagopalan (1989) 
compared several authoring systems In terms 
of their power, ease of use, and productivity. 
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Some examples of authoring tools/systems are 
Authorware Professional™, IconAuthor, 
Quest, PCD3, HyperCard, and Linkway. 

d training: The teacher-developers and subject 
matter experts must be trained In the 
principles of instructional design with 
emphasis on CAI to ensure that the software 
developed will be Instructlonally sound. The 
teacher-developers must be rigorously trained 
by the authoring systems expert through a full- 
time ln-servlce course that would require that 
they produce courseware to be used In the 
classroom. 

d) design/ development teams: The 
participants In a courseware development 
effort will be divided Into development teams. 
Figure 1 shows the composition of such a team. 
In this model, each Instructional 
designer/coordinator and the authoring 
sys* ;ms expert would be overseeing five small 
teams consisting of a teacher-developer and a 
subject-matter expert. The teacher-developer 
of each small team, together with the subject 
matter expert, will develop a courseware 



module for a topic or sub-topic in the 
curriculum related to their specialities and/or 
Interests. The module that they will develop 
will be between 15 to 30 minutes in length with 
an emphasis on helping teachers to deliver 
Instruction in the particular subject area. The 
team members will meet to discuss any 
problems encountered as well as show the 
coordinators their current progress in their 
work at specified times every week. These 
sessions will be used as a peer group formative 
evaluation sessions to generate better ideas and 
techniques of developing courseware. 

e) pilot testing, evaluation and identification 
of teaching strategies: At the end of a certain 
time length (e.g., three months), the teacher- 
developers should try out and conduct a pilot 
test of their products either in the classes that 
they are teaching or with the help of their 
colleagues. During this time, together with the 
subject matter experts, they will identify the 
strength and weaknesses of the courseware and 
the most effective strategies for 
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Figure 1. An example of a software development team. 
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teachers to use the software created. The 
evaluation would utilize qualitative and 
observational methods as well as traditional 
quantitative methods of evaluation. 

f) training other teachers to use the 
courseware: After the major and obvious bugs 
or faults of the courseware have been Identified 
<uid fixed, the teams must meet together to plan 
an in-service teacher training program to 
support the use of the developed courseware. 
The teachers will be trained by both the subject 
matter experts and the teacher-developers to 
use the developed courseware effectively. 

The first group of teachers that will be trained 
should consist of the teachers teaching the 
relevant subject areas from the same schools as 
the teacher developers. This will ensure that 
there Is back-up expertise In dealing with the 
computer or courseware If problems are 
encountered by the classroom teachers in the 
school. 

Figure 2 shows the phases of the model's 
suggested development process. The 
development process has been divided into the 
development phase and the implementation or 
dissemination phase. In the development 
phase, the selection of the teachers, curriculum 
areas for courseware development, and 
software refinement should be planned, 
organized, and monitored by a central agency 
responsible for computer education In the 
country. The courseware development will 
actually take place at regional or local levels. 



This would instill within the teams a sense of 
ownership and personal Investment which is 
crucial In nurturing and the acceptance of 
innovations in education (Rogers. 1983). The 
central planning and monitoring is important 
to ensure that development Is guided towards 
the policies and aspirations of the educational 
system. It also helps to promote the sense of 
Importance of the courseware developed and 
encourages its use by teachers at the school 
level. 

The implementation and dissemination phase 
should be planned and organized at the local or 
regional level and monitored by the central 
agency. Locally developed dissemination and 
teacher training programs would be responsive 
to the particular interests and characteristics 
of the teachers Involved as well as encourage 
innovations in the use of the courseware 
developed. Programs developed and taught by 
local staff will support professional growth 
and leadership at the local level Local trainers 
would also be able to offer on-the-job support 
as teachers try to Implement what they have 
learned in training experiences. 

Figure 3 shows the set-up suggested by the 
model and the links between the central agency 
and the local/ regional development teams. 
Collls and Olivelra (1990) described the 
advantages and limitations of different types 
of computer-related educational policies. They 
concluded that emphasis 
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Figure 3. Links between the central monitoring agency 
and regional development teams. 



should be placed on supporting a blend of 
centralized and localized policies to support 
the development and implementation of 
educational computing. 

Conclusion 

The importance of good software to support the 
diffusion process of computer technology in 
educational settings is beyond doubt. However, 
good software that works in a particular 
educational setting does not usually mean that 
it is also portable to other educational 
environments (Ely, 1990; Murray-Lasso 1990). 
Thus, developing nations have to find cost- 
effective ways of producing high quality, 
relevant software for their own education 
systems. 

It is hoped that the systematic courseware 
development model described above will help 
in providing guidance and suggestions for 
educators in developing nat' s as they 
attempt to introduce CAI into th educational 



systems. Of course, this paper is only a brief 
overview of what will eventually be a more 
detailed generic mode! for use in developing 
countries. More* detailed sequences of steps and 
considerations need to be specified to fully 
implement a systematic approach to 
courseware development in a specific 
developing country due to the unique cultural, 
political, and social features of each country. 
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THE ROLE OP THE COMPUTER IN SCIENCE FAIR PROJECTS: 
CURRENT STATUS AND POTENTIAL 



Mary S. Trainor Los Alamos National Laboratory 



ABSTRACT 

The need for more students to enter the field of 
science is acute in the nation, and science fair 
projects provide a motivational mechanism to 
entice students into pursuing scientific careers. 
Computers play a major role in science today. 
Because computers are a major source of 
entertainment for our children, one would 
expect them to play a significant role in many 
science fair projects. This study investigated 
current and potential uses of computers in 
science fair projects and incorporated an 
informal case study of scientists, teachers, and 
students involved in science fair projects from 
a highly scientific community. Interviews, a 
survey, and observations were conducted. 
Results indicated that most projects either do 
not use or inadequately use computers and that 
a significant potential for more effective use of 
computers for science fair projects exists. 

INTROLJCnON 

Science fair projects provide a mechanism to 
attract young people to enter science and 
engineering fields. Workforce projections 
reflect a drastic decline of available 
professionals in these fields; thus, science fair 
projects offer valuable resources for studying 
how to motivate and encourage potential 
scientists. In most elementary, Junior high, 
and secondary schools, students receive 
encouragement and some help to prepare 
original projects. Computers now play a m ,or 
role in science and engineering research, yet 
they do not seem to play a central role In 
science fair projects in spite of the fact that 
computer games are a primary source of 
entertainment for many students. 

This paper explores the subject of the role of the 
computer in science fair projects. The specific 
research questions asked were the following: 

How are computers being used by K- 12 students 
for science fair projects? 

How can students be helped to use computers 
more effectively for their projects? 

The hypothesis being tested is that computers 
are inadequately used and. In some cases, 
misused in science fair projects, he methods 



used in this study were interview and 
observation. A small sample of students, 
teachers, school administrator, and scientists 
(science fair Judges) were nonrandomly selected 
and systematically asked to provide 
information on how they feel computers could 
be used and have been used at science fairs. In 
addition, suggestions from the author 
regarding feasible potential application of 
computers to projects are provided. 

This study is -not intended to be an all- 
encompassing survey of science fair projects 
but rather Is case study. The goal is for this 
study to prompt increased interest in the use of 
science fair projects to attract students Into 
scientific careers and to promote the 
appropriate use of computers for science fair 
projects. In addition, vendors might consider 
the development of new softw are packages, 
tailored to assisting students with projects as 
an important market. 

BACKGROUND: SCIENCE FAIR PROJECTS 

A science fair project is an experiment or 
investigation designed and conducted by a K- 12 
student and then reported in a research paper 
and/or exhibit and presentation. One science 
fair project publication for students (Van 
deMan and McDonald, 1980) states, "All 
outstanding projects usually have one thing in 
common - They are the result of creative 
thought." The publication further elaborates 
with the following Information: 

•It is deciding if you are willing to make the 
commitment to see your project through 
from start to finish. 

•It is choosing a topic that seems interesting 
to you and something that you would like to 
know more about. A good way to start is to 
ask a question that can be answered only by 
experimenting. 

•It is formulating a purpose for your project 
and making hypotheses about the outcome 
of your experimentation. 

•It Is experimenting to test your hypotheses, 
making observations, recording your data. 
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•It is analyzing your findings, drawing 
conclusions, preparing your paper, exhibit, 
presentation, and presenting your project to 
an audience of Judges, the public, and your 
classmate?. 

Kennlson's handout (1990) provides a more 
flexible approach to problem selection, for it 
allows two types of projects: 

•Investigation of a Problem. Construct a 
research question and then design an 
experiment. 

•Construction of a kit or model, or putting 
together a collection. 

The criteria listed for Judging of the projects 
are (1) creative ability, (2) scientific thought, (3) 
thoroughness, (4) skill, (5) clarity, and (6) 
dramatic value. 

The difference between true scientific research 
and science fair projects must be noted. To a 
great extent, science fair projects teach how to 
do such projects and not how to do research 
projects that scientists perform. In scientific 
research, much collaboration and study occur 
before selecting a research problem. Research 
problems also tend not to be selected in 
isolation, but rather as part of larger ongoing 
efforts resulting from years of concentrated 
study. In scientific research there is rarely 
strict adherence to the scientific method, as 
described in science fair publications (e.g. Van 
deMan and McDonald). The sequence of steps 
taken Is not linear, for iterations occur within 
steps and steps are sometimes skipped and then 
revisited later. The time scale is rarely as 
predictable as necessary for a science fair 
project. 



BACKGROUND: THE ROLE OF COMPUTERS Of 
SCIENCE AND EDUCATI?" 
In the past few decades, computers have 
emerged as being tools central to scientific 
research. Thirty years ago, computers were 
often used for data analysis. Today, computers 
are used as tools during every stage of the 
research process and are used for literature 
searches at the beginning of the process. They 
are used for electronic communication and 
collaboration among researchers. Word 
processors, spreadsheets, project management, 
and graphics tools are used to prepare 
experimental designs and proposals. 
Computerized instruments or automated 
control systems are often involved in data 
collection. Statistics packages and other 
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software tools are central to the compilation, 
analysis, and reporting of the research results. 
Aside from the role of computers to scientific 
research, computer science is a field itself. 
Computer languages, graphics, interface 
design, hardware, data structures, artificial 
intelligence, and algorithms are Just of few of 
the computer science disciplines that can be 
delved into for science fair projects . 

The role of computers In education is changing 
rapidly but still lags behind computer usage to 
science, which directly affects the role of 
computers in science fair projects. Most 
schools now have some type of computer 
laboratory and users courses. "At Cincinnati 
Country Day School two principles underlie 
the school's commitment to restructuring: 
First, that learning is best achieved by doing, 
and second, that technology - better than 
anything else so far - promotes principle 
number one." (See Pearlman, 1991.) Many 
teachers have a personal computer in their 
classrooms, but few are heavily used because of 
a lack of time in the teacher's busy schedule, 
lack of hardware or software support, or lack of 
adequate training on the part of the teacher. 
The bulk of computer use in the schools Is to 
the computer labs during formal class time. In 
Los Alamos, the computer labs in both the 
middle and high schools are heavily booked 
with classes. Because of budgetary constraints, 
schools rarely have open computer labs where 
students can use the computers for projects 
during their free time. 

Many schools envision that the application of 
computers to learning is limited to learning 
programming, applications package s (e.g., 
word processors, spreadsheets, databases), and 
computer-based instruction (CBI). CBI has 
existed as a field for over thirty years, yet few 
schools have state- of- the art CBI courses. The 
potential use of computers as tools for 
learning, however, is \nuch greeter than is 
commonly perceived (Ai.urews et al , 1988). The 
potential uses range from record keeping to 
class presentation and from gaming and 
simulation for individuals and groups to 
electronic communication with experts and 
students at other schools. 

A very recent study of the California schools 
revealed some relevant data on computer usage 
(Main and Roberts, 1990). A written survey was 
administered to 1000 schools and 484 were 
returned. Some of the data highlights follow: 

•An average of 40 computers are at each school 
site (slightly more than one per classroom or 
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one per teacher). 

•Computers outnumber TV monitors by more 
than five-to-one. 

•The most common location for the computer 
Is In the classroom, with the laboratory 
configuration a close second. 

•The most commonly used computer 
applications are word processing and 
Individual drill and practice or tutorial. 

•Over half of the teachers are rated as being of 
limited proficiency or unskilled for both 
personal productivity and instruct ional use 
of computers. 

•Ninety-one percent of the schools perceived 
that integrating technology into the 
curriculum was the greatest staff 
development need. 

This data, from an educationally very 
progressive state reveal that computer 
hardware is universally in place, but the use of 
the hardware and state-of-the-art applications 
is not optimal. 

Computers are useful tools to facilitate 
learning, solving problems, organizing or 
retrieving information, doing simulation, etc. 
but they are only tools. A good education can be 
obtained without knowledge of computers, but 
computers have become instrumental in many 
Jobs. As with science fair projects, computers 
can potentially help students organize and 
present their work more effectively and 
efficiently. Students need to obtain an accurate 
view of how computers are used in the 
workplace; science fair projects can help 
achieve this objective. 

METHODS 

This study involved interviewing and/ or 
administering a survey to the following: 

•two middle school science teachers, both 
heavily involved in helping students to 
conduct quality experiments 

•one high school computer science teacher, 
who teaches a course on using computers in 
science fair projects 

•one school principal, who chairs the Science 
Fair Committee, 

•two scientists who had Judged at several 
science fairs, and 



•three stx/ nts who had done at least two 
projects in the past (one each from 
elementary, middle school, and high school 
levels). 

Subjects were chosen systematically, on the 
basis of heavy involvement with past science 
fairs. Subjects were not chosen, however, on 
the basis of computer expertise or experience. 
Interviews were conducted in an informal 
fashion, but a list of questions was use d as a 
guide (see Tables 1. 2. and 3). All interviewees 
were from Los Alamos, New Mexico, a unique 
small town with a very high percentage of 
resident scientists. Therefore, the interest in 
science fair projects is keen among students, 
parents, and teachers. Competent, motivated 
Judges are readily available! 

RESULTS AND DISCUSSION 

The surveys and interviews revealed a range of 
experience and opinions regarding computer 
use in science fair projects (see Tables 4 to 8). To 
provide equal weighting of all opinions, the 
results are presented, without sorting the 
student's or instructor 1 s opinions from those of 
the scientists. Highlights of differences 
between categories are presented below. All of 
the collected data is not reported here because 
of space, but the tables contain the most 
representative feedback. Data on computer use 
in 1991 science fair projects is not yet available 
but will be provided at the time this paper is 
presented. 

One teacher felt that motivating students to 
pursue scientific careers was only a minor goal 
of science fair projects. He oaw the primary 
goal as giving them a long-term learning 
experience that requires dedication and follow 
through. Science fair projects need to Le seen as 
an opportunity for all students (not Just those 
interested in science as a career). 

A high level of computer literacy prevailed 
among the subjects, for all of them had laken a 
computer course and used a computer fairly 
regularly, except for the elementary school 
subject. He had only used a computer for 
computer games, a little word processing, and 
math drill and practice, although he had been 
exposed to several different computer 
applications by observing his parents . To 
motivate more students to use computers and to 
represent the real world more accurately, the 
middle school and high school students both 
had taken computer science courses that had 
included using word processing packages and 
some programming. The middle school has 
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recently moved from emphasizing 
programming to teaching applications. They 
help students use spreadsheet, graphics, and 
database packages. Drawing a box with a 
simple program in BASIC was a turnoff to some 
students, whereas drawing sophisticated 
graphics or seeing a spreadsheet accommodate 
changes instantly are more fun. 

The Los Alamos Middle School, to an effort to 
encourage science fair projects, has recently 
enacted a new program in which the teacher for 
gifted students gives individual help to students 
doing science fair projects. Partially because 
there have recently been very few projects in 
the computer science category, the high school 
has added a self-paced, individualized 
computer science course for students working 
on science fair projects. The students who sign 
up for this course generally are working on 
computer science or mathematics projects, as 
opposed to the other sciences. Therefore, the 
chances for winning a prize are greater in 
computer science than they are in physics or 
chemistry. 

One knowledgeable interviewee claimed that 
computers are used In approximately 50 to 60% 
of the science fair projects above the sixth 
grade level, Two other interviewees estimated 
computer involvement as much lower, 10% and 
25%. Almost all of the uses of computers for 
these projects, however, is in the word 
processing or presentation graphics areas (see 
Table 5). Table 6 shows the potential uses of 
computers in science fair projects. It was 
interesting to note that the students 
interviewed perceived that they used the 
computer to its maximal potential in their 
projects, yet relative to the list on Table 6, they 
were far from fully exploiting the computer. In 
the elementary grade levels, the use of 
computers is less than that for the higher 
grades and again is used for presentation only 
and not for analysis. 

In terms of the type of hardware used, an 
interviewee hypothesized that students who 
had a Macintosh computer at home were more 
likely to use it for their project because the 
interface is easier to use without parental *ielp 
than that of other computers. 

Many of the elementary school age children 
have their parents do the presentation graphics 
and text for them. The Judges interviewed did 
not consider this as misuse, unless the parent 
edited the child's text while typing the 
presentation. Unfortunately, most parents are 
unable to resist editing their children's work 



and thus misuse does occur. The Judges agreed 
that the way to identify the projects in which 
parents had too large a role In from those the 
child did Independently, is through the 
interview. At the time the Judges are going 
around examining the project displays, they 
talk with the children and ask for 
explanations. If the child cannot explain why 
the outcome was interesting or why the 
experiment was set up the way it was, then the 
misuse is readily perceived. The only type of 
misuse named by the Judges interviewed was 
too heavy parental involvement. 

Table 5 presents several examples of science 
fair projects in which the computer was used 
for something other than presentation 
graphics and/ or word processing. Several of 
these same projects were mentioned by more 
than one interviewee, suggesting their 
uniqueness. One interviewee noted that role 
modelling played a big role in science fair 
projects. Many students started participating 
in projects in the elementary grades and each 
year would study other student's projects. When 
they saw a project they thoughts exceptional, 
then they tended to try to model parts of it the 
next year. This is one major way that students 
can learn about the range of possible uses of 
computers. This process, however, is currently 
an Informal one done byjust a few students and 
could be incorporated formally Into the science 
curriculum. 

How can a student's perception of how 
computer use in science become more accurate? 
Table 8 presents a list of methods that those 
interviewed cited as ways of increasing and 
improving computer usage In the projects. This 
list contains several significant areas of 
development for university education 
departments, schools, and vendors. The answer 
to the question, "Do you think that the use of 
computers in science fair projects should 
increase?" was yes from all the subjects. One 
teacher said, "How are kids going to get into the 
21st century if they do not have more exposure 
to computers? This is only going to happen if 
our budgets for computers increase." 

SUMMARY AND IMPLICATIONS 

From the data collected in this informal study, 
we can conditionally accept the hypothesis: 
Computers are Inadequately used and, in some 
cases, misused in science fair projects. The data 
did indicate that computers are Inadequately 
used to science fair projects. The issue of 
misuse aroused some controversy, perhaps just 
because of the variable interpretation of the 
word misuse. The majority Sid not see 
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examples of misuse of computers in terms of 
correct application of the technology in the 
projects they had obsewed. However, when 
asked about parental involvement, some 
interviewees saw misuse playing a role. 

The underuse of computers in science fair 
projects fell into two categories: (1) The 
students whose project presentations could 
have been improved through the use of a 
computer and \2) The students whose project 
content could have been improved through the 
use of computers for analysis, control, or 
simulation. The interviewer's general 
impression of most interviewees is that the 
wide range of ubes of computers for science is 
not known in the schools. If it is known (for 
example by the high school computer science 
teachers), often the range of possible uses 
cannot be shown to the student because of 
hardware (budget) or schedule constraints. The 
students who used computers for analysis or 
control (Table 5) were generally those whose 
parents were very computer literate and helped 
them and those who had a well-equipped 
computer system at home. Schools seem to be 
adequately informing students about the uses 
of computers for word processing, 
spreadsheets, and presentation graphics. 
Schools do not seem to be able to provide 
adequate hardware or software to students 
wishing to use computers in this way for their 
projects. In addition, schools seem unable to 
provide students the exposure to or 
opportunities for data analysis via computers 
not because of a lack of computer literacy 
among teachers, a commonly cited reason, 
rather because of a lack of an adequate 
computer budget. Students need the following: 

•More free time on computers, therefore more 
computers, software, and computer-aids or 
teachers to advise individuals. 

•More exposure to the range of uses of 
computers in science through examples. 

•More software tools tailored for doing science 
fair projects. For example, a statistical 
package in which one can enter he data, 
obtain averages and standard deviations and 
basic plots. 

•Computer-based Instruction lessons that help 
in getting started This instruction might 
Include game-like packages to challenge the 
students into narrowing down their area of 
Interest and then focusing on a problem. 

•More computer networking at the schools. 



Such networking would provide the following 
types of opportunities not currently 
available: (1) electronic consultation with a 
remotely located expert in the field they are 
researching, (2) accessing information from 
databases that are not available locally, and 
(3) accessing mainframe computers that have 
much greater power than is locally available. 

The computer is a tool that can enhance the 
quality of a science fair project. Computers are 
not seen as a central focus of science fairs but 
rather as aids. Increasing and enhancing the 
use of computers in science fair projects is, 
however, an opportunity to further prepare our 
children for the workplace of the future. 
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NAME 

AREAS OF EXPERTISE 

1. What Is your experience with science fair 
projects? 

2. Have you Judged science fair projects? If yes, 
discuss the experience briefly. 

3. What do you believe is the main benefit of 
science fair projects? 

4. Do you use a computer? YES NO If yes, for 
what and how often? 

5. At your school, how many students receive 
computer training? Of what does that 
training usually consist? 

6. At your school, is there any encouragement 
for students to use computers for their 
science fair projects? YES NO If yes, 
describe: 

7. Do you see it as beneficial for students to use 
computers in science fair projects? YES NO 
If yes, how? 

8. In the science fair projects that you have 
been in contact with in the past year, what 

percentage used computers? Give 

some examples, if you can, of specific 
projects that did use computers: 

9. Can you give some examples of projects 
where computers were misused? 

10. Do you know of any learning resources for 
students to use to learn how to use a 
computer to aid them in their projects? YES 
NO If yes, what are they? 

11. Do you think that the use of computers in 
science fair projects should Increase? YES 
NO If yes, how? 

TABLE 1 INTERVIEW QUESTIONS FOR 
TEACHERS. ADMINISTRATORS 



NAME 

GRADE LEVEL 

1. How many science fair projects have you 

done? What was/were the 

title(s)? 

2. What do you think that you learned from 
doing your projects? 

3. Do you use a computer? YES NO If yes, how 
and how often? 

4. How do you think that a computer is used in 
doing science? 

5. Have you ever had a computer course? YES 
NO If yes, what did you learn? 

5. Did you use a computer in your science 
project (s)? YES NO If yes, please describe the 
software, hardware, and type of use : 

6. Would you like to have used a computer more 
in your project(s)? YES NO If yes, for what? 
If no, why not? 

7. Can you tell me examples of other students' 
projects where a computer was used very well 
and very poorly? 

8. How do you feel that your parents and/or 
your school and/or software companies 
could help students use computers more 
effectively In science fair projects? 

TABLE 2 INTERVIEW QUESTIONS FOR 
STUDENTS 
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NAME 
FIELD 

1. What Is your experience with science fair 
prelects? 

2. What do you believe Is the main benefit of 
science fair projects? 

3. What was your responsibility as a science 
fair Judge? 

4. What guidelines were you given for your 
Judging? 

5. How did you Judge? (What did you look for 
the most?) 

6. Do you use a computer? YES NO If yes, for 
what and how often? 

7. Do you see It as beneficial for students to use 
computers In science fair projects? YES NO 
If yes, how? 

8. In the science fair projects that you have 
been In contact with In the past, what 

percentage used computers? Give 

some examples, if you can, of specific 
projects that did use computers: 

9. Can you give some examples of projects 
where computers were misused? 

10. Do you think that the use of computers in 
science fair projects should increase? YES 
NO If yes, how best do you think this could be 
accomplished? 

TABLE 3 INTERVIEW QUESTIONS FOR 
SCIENTISTS 



•To develop a project from beginning to end; the 
satisfaction of completing a project 

•To gain practice In presenting the project to 
judges and in so doing develop poise and 
confidence 

•To learn the scientific method In the research 
of specific areas/problems of Interest 

•To allow students to delve into an area of their 
own interest, in contrast to the curriculum of 
the schools, which necessarily allows for little 
flexibility 

•To enable students to see science more like It 
really Is in real life, not like it is In the 
classroom 

•To provide an opportunity for academically 
oriented students (as opposed to sports 
oriented students) to excel at something and be 
rewarded for it • 

•To learn how to collect and organize data 

TABLE 4 RESULTS: MAIN BENEFIT OF 
SCIENCE FAIR PROJECTS 



•Word processing (report preparation) 
•Spreadsheets 

•Computer program for problem solving 

•Presentation graphics 

•Microcomputer control systems 

TABLE 5 RESULTS: CURRENT COMPUTER 
USAGE IN PROJECTS 
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•Accessing databases 
•CAD 

•Electronic communication and collaboration 

•Computer-based Instruction on the scientific 
method 

•Data analysis (statistical packages) 
•Record keeping 

•Outlining; organizing information that is 
collected 

•Simulation 

TABLE 6 RESULTS: ADDITIONAL USES OF 
COMPUTERS IN PROJECTS 



•The position of the moons of Jupiter 

•Experiments on hair strength and the effect of 
microwaves 

•The concept of chaos 

•Computer-controlled robotics 

•Computer-controlled hand 

•Computer imager 

TABLE 7 RESULTS: EXAMPLES OF PROJECTS 
USING COMPUTERS FOR ANALYSIS AND/OR 
CONTROL 



•Making more computer courses available 

•Having open computer labs where students can 
use the computers during free time for their 
projects 

•Having software packages tailored for science 
fair projects (e.g., for problem recognition and 
definition/narrowing, for var iables 
identification) 

•Providing easy-to-use presentation graphics 
software 

•Science teachers showing example 
applications of computers in science, from 
which students can learn about the broader 
potential for computers 

•Modeling of possible solutions, varying of 
parameters 

•Assigning computer-educated volunteers to 
children who need help with computers 

•Computer teacher offering sessions to give 
children ideas on how to use computers for 
science fair projects 

TABLE 8 RESULTS: EFFORTS THAT CAN 
IMPROVE/INCREASE COMPUTER USE IN 
PROJECTS 
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CAI IN LEARNING ENGLISH AS A SECOND LANGUAGE (ESL) 



Lih-Juan ChanLin, The University of Georgia 



Abstract 

The roles of computers in the field of teaching 
English as a second language (ESL) have been 
discussed from different language learning areas: 
reading, grammar, writing, and 
listenin^speaking. As a role of tutor, the tutorial 
program provides different levels of linguistic 
knowledge. As a role of tutee, the programming 
language allows students to use knowledge of 
language to communicate with computers. In 
addition, the computer also works as an editor, 
an advisor, an idea generator, and an expert in 
various computer applications. Concerns of 
integrati ng technology in ESL learning within 
various learning areas are stressed in this study. 

Introduction: 

In many countries, English learning is 
considered as an important issue in school 
curriculum. Learning English not only 
introduces students to English speaking 
culture, heightens awareness and 
comprehension of one's native tongue, but also 
serves the nation's needs in commerce, 
diplomacy, defense and education. When 
introduced to English as a second language 
(ESL), the learners have the opportunities to 
explore a new system of thinking within the 
new cultural context. At an advanced level, it 
also provides the chance of exploring new 
literary and political perspectives (Savlgnon, 
Curtain & Pesola, 1988). With learning 
English for this purpose, the use of computers 
in the field can be either direct instruction, 
focusing on the form of message presentation, 
or indirect language instruction through 
various subject areas. The use of the computer 
as an instructional tool has drawn a lot of 
attention during recent years. Many 
curriculum specialists have been designing 
software for a variety of instructional 
purposes, such as learning writing skills, 
reading skills, listening comprehension, and 
grammar. The role of the teacher is also seen 
to change from an information provider to an 
organizer and classroom manager (Cheung, 
1987). The purposes of this review are to: 
identify the role computer is playing in the 
field of teaching English aa a second language, 
identify possible problems existing in this 
approach, and forecast the trend of computer 
learning in ESL in the future. 



Major linguistic aspects that constitute 
language learning are basically vocabulary, 
spelling, and grammar, but there are also 
attempts on cultural awareness and 
soclollngulstic appropriateness, and discourse 
In most .anguage learning. Dalgish (1990) 
suggested that the computer can be applied in 
various aspects of linguistic knowledge with 
focus on the linguistic nature of context. The 
linguistic concept described by Le (1989) 
contains three interacting factors: syntactic, 
semantic and discourse levels. Several 
computer programs were developed based on 
these factors to influence children's encoding 
task and to establish the learning context. 
Based on the different linguistic purposes that 
computer assisted instruction (CAI) is intended 
to achieve, various modes of presentation are 
discussed as followed. 

Teal for Teaching Reading /Language art: 

Reading is a multileveled, interactive, and 
hypothesis-generating process in which 
readers construct a meaningful representation 
of text by using their knowledge of the world 
and language (Mustapha, 1988). Results of 
various research studies on the reading process 
and on the view of reading comprehension 
have indicated that reading is an interactive 
process between the readers and the context 
(Mustapha, 1988, Preislnger, 1988: Hedley, 

1985) . With the notion of providing interaction 
as a way to facilitate students' thinking 
process, the computer has been used in many 
ways in reading and language arts (Balajthy, 

1986) . It provides the function of drill and 
practice of subskills , such as electronic flash 
card, vocabulary quizzer, crossword puzzle, 
sentence/phrase association. The purpose of 
computer drill and practice is to provide novice 
learners with the smooth continuity and 
automatic process Important to performing a 
complete skill essential for reading (Dreyfus & 
Dreyfus, 1985; Balajthy, 1987, Nyns, 1988). 

In teaching reading, the different reading 
speeds among learners can be considered as a 
major problem. Especially, for non-English 
speaking learners, it is necessary to employ 
some strategies to comprehend the meaning of 
new words. The strategies that are often used 
are inferring the meaning of new words in the 



ERIC 



mother tongue, deciding on the importance of 
certain words for the global comprehension of 
a text, and dealing with compound and affixed 
words (Palmberg, 1988). Due to Individual 
differences in the ability of using these 
strategies, comprehension time for different 
students varies from one student to another. 
Consequently, among a class of students, 
reading speed might vary greatly. In order to 
conduct group classroom activities, faster 
readers sometimes spend time waiting 
resulting boredom. On the other hand, lower 
speed readers who are never allowed to read 
through a text, will always feel frustrated. To 
increase the reading speed of every individual 
student with various starting points, and 
without forcing either the faster reader or the 
slower reader to compromise with each other, 
the computer Is the ideal medium which offers 
such control. In addition to reading speed, 
allowing the adaption of different modes of 
reading also stresses the flexibility feature of 
using computers. 

Another important Issue in teaching reading is 
the syntax and text structure. Grammatical 
structure can help in comprehending text 
materials. However, non-English speaking 
learners usually have problems interpreting 
pronouns, articles, and conjunctions or logical 
connectives correctly. The solution to this 
problem is to provide sufficient practice 
(Nuttal, 1982; Palmberg, 1988). Computers can 
advantageously supply this guided practice 
which is necessary for skill development. 
From another approach, in order to develop 
predictive reading by consciously using 
knowledge about the structure of the text that is 
being read to guide comprehension, the learner 
control and guided practice are both considered 
very Important, and computer is the ideal tool 
to serve this purpose (Higgins, 1984; Palmberg, 
1988). 

Because computers can be easily programmed 
to recognize and respond appropriately to true- 
false, multiple choice, matching drills, they 
serve as an effective tool to diagnose, prescribe, 
present tutorial instruction, and drill 
reinforcement, and furthermore, to monitor 
student achievement. However, as concerned 
by Balajthy (1987), computers can be used or 
misused. Instead of considering CAI as an 
"end", one should consider it more as a "tool". 
For more advanced learning in reading, one 
needs to develop higher level reading 
comprehension skills (Kleinmann, 1987). 
When the purpose of monitoring the subskllls 
is achieved by using CAI materials, students 
need to move on toward more complex human 
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activities to achieve the heart of language 
function, meaningful communication. 

Learning of a language starts with learner's 
interest in relating learning materials to the 
everyday needs of communication. As pointed 
out by Palmberg (1988), the issue of language 
needs is of the special importance to learners 
when it comes to beginners. His concern is 
more on the content Itself to achieve the 
motivation purpose. A simulation model in 
computerized foreign language is suggested by 
Wu (1984) as an approach to increase the reality 
of learning materials and encourage active 
involvement in reading for understanding. 
Since the materials presented satisfy the 
criterion of language needs, relevant to young 
learner's real life activities, they promote 
vocabulary learning. The notion is that 
comprehensible input of the instructional 
material that Is interesting, relevant, and at an 
appropriate level of complexity is crucial to 
second language development (Krashen & 
Terrell, 1983). 

Described by Klelnmann's study (1987), In the 
CAI approach of reading achievement, the CAI 
environment can facilitate an Informal Intake 
process and formal learning process. As an 
informal intake environment, the CAI 
Instruction allows input of reading material to 
be matched with the particular level of 
students. The materials can be chosen In 
different levels according to the individual 
needs. As a formal learning environment, the 
CAI instruction provides the necessary 
interaction and feedback which is equivalent 
to the classroom learning environment. 
Whether providing a formal learning 
environment, or providing a informal learning 
environment, the attention of developing CAI 
should be paid to creating meaningful 
communicative interaction with learners. 
Moreover, how to facilitate the development of 
self-monitoring skills through the provision of 
learning strategies within the learning 
materials Is also necessary to be considered. 

Grammar Software: 

Knowing grammar is a metacognitive skill that 
allows people to integrate new knowledge with 
the complicated nature of the language they 
have mastered. Learning grammar is to relate 
different linguistic rules, and apply in different 
categories of situation. From an historical 
perspective, computer assisted instruction 
began in the 1960's during the time when 
theories of structural linguistics were giving 
way to those of transformational grammar 
(Jamieson & Chapelle, 1984). As described, 
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CALL (computer assisted language learning) 
grammar lessons were developed to use 
computerized grammar drills and to give 
students out-of-class practice. The content of 
those grammar lessons often focused on more 
mechanical aspects of language. The exercises 
were mainly mechanical, and few were 
meaningful or communicative. The same 
situation is found to the current approach of 
grammar software. Most ESL software claims 
to teach grammar is of a traditional 
prescriptive nature and is often in a drill and 
practice format with inadequate ability to 
Judge student's response. There is great concern 
about the poor quality software flooding the 
market(Dalgish, 1985a,1985b, 1987). Among 
the problems noted are: inappropriate lesson 
content, poor documentation, errors in format 
and content, improper feedback, etc. (Dalfiish, 
1987). * 

This traditional approach to language teaching 
has been criticized by recent supporters of a 
totally communicative approach to language 
teaching. The computer should be seen as a 
useful tool for language study by providing the 
students with practice in communicative 
aspect of a language. Ideally software should 
be designed with consideration of judging 
student's answer, and analyzing student's 
answer in order to provide relevant feedback to 
help them learn (Hart, 1981; Jamieson & 
Chapelle 1984). Instead of simply providing 
feedback of "right" or "wrong", an intelligent 
computer program should be able to identify 
student's misconception that led to the 
production of a wrong answer, or a wrong type 
of answer. If well designed, good quality drill 
and practice may have a place after all in a 
fully integrated curriculum, as long as the 
computer allows students to use them 
functionally. 

From one approach, grammar software of drill 
and practice sets rules as an essential requisite 
to grammar learning. In another approach, 
structured grammar can be acquired from a 
descriptive perspective in order to motivate an 
appreciation of the rules of grammar through a 
understanding of them. Freyd (1988) suggested 
the use of the whole language approach through 
logo activities. Programing languages with 
natural language manipulation can formulate 
rules of grammar based on the student's own 
knowledge of English. The nature of natural 
language and lexical categories in particular 
can be generated by these programming 
activities. 



Computer la Writing: 

Writing can be. considered as a sequence of 
processes, including brainstorming, 
production, and revising. By viewing writing 
as a process instead of focusing on the products, 
a lot of researches addressed the importance of 
the writing process and use this approach to 
apply computer assisted instruction to writing 
(Burns, 1980; Daiute.1985; Selfe & Wahlstrom, 
1982; Shostak.1984; Wresch, 1983). The stages 
involved in writing can be categorized into: 
prewrittog, composing, rewriting and editing. 
At the prewrittog stage, the process is usually 
focused on invention, encouraging learners to 
concentrate on the topic. When students are 
guided to concentrate on the flow of a narrative 
example, they are much easier to develop 
higher-level thinking strategies, which means 
it is rrore able for them to organized their 
thoughts in an elaborated way. In cognitive 
psychology's investigation of children's 
writing and reading, studies have shown that 
children, when faced with high-level cognitive 
tasks, tend to focus on lower-level processes 
such as decoding to reading, or spelling and 
handwriting in composition (Shostak, 1984). 
Several kinds of computer software can be used 
in the prewritlng stage to free learners from 
attending to low level processes and focus on 
higher-level thinking process. StoryMaker 
(elementary level) is a typical example for this 
purpose. In addition, software like 
Wordsworth II (Selfe, 1984), and MaxThtok 
(Hershey, 1985) are designed to guide students' 
thinking through of structured concepts. 
Although this type of software is mostly used 
with native English speaking students, it could 
be a potential approach for non-English 
speakers to teaching writing to ESL class. 

Word processors 

In the composing stage, computerized word 
processing can be considered as the most 
convenient tool in writing. The main 
advantage of using a word processor is that it 
makes the mechanical aspects of the drafting 
and revising process simpler to attend to. 
Because of this advantage, it has become an 
important writing aid for almost any field at 
any level. A good, simple powerful word- 
processing package can foster a sense of student 
control over the computer (Dalglsh, 1987). As a 
productive tool, the word processor can 
produce professional-looking final 
manuscripts. With a word processor, users can 
learn to mold and shape their writing and 
catch ideas as they flow. Due to this reason, a 
lot of learners are motivated not only to write 
and to revise but also welcome others to read 
and talk about the writing. Also, when using a 
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word processor to write, users are frequently 
more open to the idea of change (Herrmann, 
1986, 198b;. Students tend to be highly 
involved in written context and gain new 
sensitivity to the flexibility of language. They 
also appear more receptive to feedback 
concerning the need for revision and editing, 
and Improve their overall writing and language 
ability. 

However, learning word processing varies 
greatly among students. The main 
disadvantage of using a word processor as 
described by Canale & Barker (1986) is that the 
software provides no means for the users to 
keep a record of the writing process, since 
people engaged in writing are not only 
interested in the product but also the process to 
produce the work. The use of computers also 
requires extra demands on students' mental 
processing by disrupting their accustomed 
methods of composing. And it usually requires 
a lot of out-of-class time for learning the skills 
(Baxter, 1988). Very often, learning how to do 
word processing Is a frustrating experience, 
even through the program is supposedly easy to 
learn. Some students deal much less well with 
frustration than the others. Nevertheless, 
students try their best to learn, because they 
believe it can make their future lives better. 
With the anxiety to the computer among 
learners, it can motivate learners to learn 
better. Students learning word processing 
usually must balance a multitude of 
interrelated and self-motivated mental and 
physical activities. When the intent is to help 
users to quickly adapt the computer as their 
writing tool, the control issues of word 
processing softwares need to be carefully 
considered in its development and design. 

Database 

Although the use of databases is largely 
overlooked in ESL classes, databases are 
considered as a very convenient tool for more 
advanced ESL writers. Database management 
packages encourage learners and teachers to 
gather, organize, modify and use information 
about language. Most of the interest in 
database software is their potential as a 
personal learning resource. However, these 
packages can be used for administrative 
purposes as well. As described by Dalgish 
(1987, 1985c), it can help writers to maintain a 
personal writing inventory of current errors, 
difficult works and idiomatic phrases. It also 
can be used by the instructor to classify and 
determine common errors cross-linguistically 
as a way to correct different types of grammar 
errors in different non-English speakers' 
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writing. By focusing on specific error types, 
and specific forms of errors within particular 
types, this approach can help users to gain 
control over their errors and focus on their 
problems in English grammar when learning 
English writing. 

Artificial Intelligence 

Artificial intelligence (AI) can be used to help 
students diagnose and proofread composition 
exercises. Some AI programs can understand 
English fairly well when it is related to a 
specific, or limited domain, and they are 
relatively intelligent about the meaning of one 
particular paragraph or series of paragraph. 
Therefore, they are capable of helping students 
to proofread or make decisions about 
grammar, clarify meaning or help students to 
evaluate their own writing in a simulated 
human fashion (Parkhurst, 1984). ILIAD is 
one of the earliest examples of an AI system in 
language (Wilson, 1986). It can use its language 
generation capability to create both 
grammatical sentences and ungrammatical 
sentences with errors typically produced by 
language learners. With inclusion of a 
semantic component the program also can help 
to ensure that the sentences are reasonable and 
coherent. The AI approach to computer 
assisted Instruction can be considered as 
potentially valuable way in language learning. 
Because of computers' capabilities of 
responding meaningfully to a wide variety of 
correct and Incorrect answers, and their 
abilities of gaining use of information from 
users and making their own analysis of input, 
a meaningful communicative purpose between 
instruction and learners can be effectively 
achieved. 

Other use of CAI In ESL writing 

Other uses of CAI like dialogue Journals, such 
as Dialog Maker, Computer Chronicles 
Newswlre, and InterLeam, are interactive word 
processing software, which connect the word 
processing with writing tools locally, nation- 
wide, or even around the world (Sayers, 1986). 
These kinds of software, that provide written 
conversation on a daily or weekly basis, can 
help ESL students learn to read and write by 
bridging from communicative abilities to 
literacy skills. At the same time, through 
actual interaction with people, the computer 
can help bring people closer together without 
regards of time and space. 

In many non-instructional settings, E-mail is 
a major use of the computer. As shown in 
Murray's study (1987), the use of computer- 
mediated communication within IBM, 86% of 
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users spend more than 10% of their working 
week using the computer as a tool for 
communication while 28% spend more than 
30% of their time on computer-mediated 
communication. This computer-mediated 
communication can also be applied In teaching 
ESL Instruction to provide collaborative 
learning experiences that foster language 
development. The use of electronic mall 
provides three important factors necessary in 
communication: speed, low cost, and ease of 
storage for future reference, modification, or 
printing (Lunde, 1990). For any language 
learning, E-mail can significantly improve 
student's reading and composition skills th<; 
target language. As an instructional tool an1 a 
communication tool as well, it can benefit ESL 
students by providing motivation to learn and 
allow students to communicate freely. 

Listening and Speaking: 

The Integration of sound into computers can be 
considered , s the most frequently Inquired in 
listening lessons and has great potential in 
language learning. Advances in voice 
synthesis and voice recognition can make a 
valuable contribution In teaching speaking 
and listening comprehension, although their 
application is currently still limited. Many of 
the physically handicapped have already found 
Important uses of voice recognition and voice 
synthesis for bypassing the need for 
keyboarding skills (Balajthy, 1987). Young 
ESL learners without learning keystrokes or 
not skilled in recognizing characters can also 
be benefit from this approach. 

Interactive technology 

Another CAI application in teaching listening 
and speaking is through the integration of 
computer and audio visual technology. There 
Is a increasing interest in combining the use of 
audio cassette, video cassette, videodisk, and 
the digital voice synthesizer for the provision 
of speech, or both speech and visual 
presentation (Marty, 1981; Rogers, 1980; 
Schneider & Bennion, 1983; Sutherland, 1987). 
It Is suggested by most language experts that 
only through real-life interactions in the target 
language can learners develop true 
communicative competence (Peppard, 1989). It 
is necessary to provide learners with context 
and control, and put learners in a real 
situation to let them Interact and experience a 
different world. By this approach, the learners 
will be able to response and get feedback with 
their own learning pace and control. 
Moreover, learners might even more freely 
communicate because with this kind of 
interaction, learners are free from fear of 
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ridicule, being rewarded for the content of what 
they say. rather than having teacher's recoil at 
their errors. Schulz and Bartz (1975) cited 
three conditions necessary for development of 
communicative competence: (1) meaningful life 
situations to practice the language; (2) the 
motivation to express; and (3) freedom to use 
and create linguistically. Interactive 
technology can provide those necessary 
condition for language skill development. 

Summary 

In ESL, computers can be applied in a variety 
of ways to facilitate the learning of language. 
The role that the computer Is playing in ESL 
learning can be as a tutor, a tutee, an editor, an 
adviser, an idea generator, a partner, an expert, 
a facilitator, and an authentic cdmmunicator. 
As a tutor, various kinds of tutorial programs 
provide different level of grammatical and 
linguistic knowledge, different content 
Information, reading and writing guidance, 
and information about the uses of strategies In 
language learning. As a tutee, some 
programming language allow students to use 
their knowledge of language to communicate 
with the computer. As an editor, the word 
processor Is a convenient tool for writers to 
edit their writing materials, and revise their 
text with less effort and less time. The role of 
the computer as an advisor is best described In 
database programs and Intelligent computer 
assisted instruction (ICAI), which present 
learning experiences from user's own database 
or from the system itself, to provide 
Information or suggestions, and encourage 
expression along directed channels. As an idea 
generator, the prewrlting software plays an 
important role In guiding student's thinking, 
and helping learners to generate idea in 
writing. In most word processing programs, 
intelligent systems, or some simulation 
programs, the computer can be considered as 
working partners to work with students and 
help students to solve given problems. In the 
Intelligent tutoring system, the computer is an 
expert in certain subject area and it 
communicates with learners In a humanistic 
way. Due to the individualization and 
Interaction characteristics provided by 
computer, in some ways, the students interest 
in learning are motivated, which Is the key 
element of the success In language learning. 
Interactive technology provides another new 
dimension of language learning with provision 
of audio and visual, and the Interactive 
characteristics, students are learning in a 
realistic environment and can communicate 
s with the Instruction in an authentic and 
meaningful way. 




Although the computer has been applied In a 
variety of areas In ESL learning, the computer 
assisted Instruction software is mainly used as 
remedial or supplementary materials. 
Computerized language learning has not yet 
reached Its optimal application. Applying 
technology usually requires long-term 
planning to integrate CAI Into school 
curriculum and existing delivery 
system/ environment. More concern need to be 
focused on the structured differences among 
different countries and areas. 

Use and design of CAI materials should take 
Into account the whole array of language 
learning In creating an authentic environment 
in reading, writing, listening, and speaking. 
New possible exploration should be opened up 
on various aspects, such as Issue of control, 
motivation design, the creation of real 
communicating opportunities, and the 
consideration of use In classroom activities as 
well as In individual learning. It Is also 
important to know that language program is 
not designed or Implemented Just to show that 
the technology car be used in this discipline. 
More understanding about the relationship 
between technology and language or language 
learning is necessary to determine what 
technology can be integrated into this area, and 
optimize its application. 
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MAGNETO-OPTHCAL DISK AS A CD-ROM DEVELOPMENT TOOL 



F. Kay Houghton and Allen L. Luettgen, 
Los Alamos National Laboratory 



Abstract 

As computer-based training (CBT) becomes 
more sophisticated, the development of the 
programs become more complex. Today's CBT 
are large multi-media systems which 
Incorporate computer-generated graphics and 
digital audio as well as text. The numerous 
computer-generated graphics, large audio files, 
and complex hyper-text structures place a great 
demand on the computer system's binary 
storage capacity (disk space). This makes a 
cr-pact disk read-only memory (CD-ROM) a 
viable delivery system. 

Because CD-ROM Is a read-only system, the 
development and delivery systems must be 
different. There are several alternatives for the 
development system, including a very large 
hard-disk and a magneto-optical disk drive. 
Recently a CBT package for radiation 
protection workers (RPTs) that was developed 
at Los Alamos National Laboratory (LANL) was 
delivered on CD-ROM. A magneto-optical drive 
was used for the development system. 

Introduction 

Computer-based training (CBT) programs are 
becoming more sophisticated. They are no 
longer electronic papers, but are large multi- 
media systems which incorporate computer- 
generated graphics and digital audio as well as 
text. The numerous computer-generated 
graphics, large audio files, and complex hyper- 
text structures place a great demand on the 
computer system's binary storage capacity 
(disk space). If the use of interactive video is 
prohibitive because of cost or other restraints, 
a compact disk read-only memory (CD-ROM) 
delivery system is a viable delivery system. 

Because CD-ROM Is a read-only system, the 
development and delivery systems must be 
different. There are several alternatives for the 
development system, including a very large 
hard-disk and a magneto-optical disk drive. 
Recently a CBT package for radiation 
protection workers (RPT s) that was developed 
at Los Alamos National Laboratory (LANL) was 
delivered on CD-ROM. A magneto-optical drive 
was used for the development system. 

This paper will discuss some reasons for 



selecting a CD-ROM delivery system and the use 
of magneto-optical disk drive as the 
development system. 

CD-ROM For Delivery of CBT 

While the laser videodisc is usually thought of 
as the vehicle for interactive multimedia, the 
CD-ROM is quickly beginning to be an option 
for multimedia delivery (Salpeter, 1991). 
While compact disk technology is probably best 
known for its Impact on the music world, the 
same technology has been used to store 
information for computer programs since 1984 
(Salpeter, 1991). There are many reasons why 
the CD-ROM should be considered as a delivery 
system for CBT. It has the advantage of a 
digital information storage which allows the 
easy transfer of data from one computer to 
another via a network or a modem. 
Furthermore, with a storage capacity of 
approximately 600 megabytes, it is equivalent 
to over 1,500 floppies (Bitter, 1988). The large 
storage capability of the CD-ROM makes it an 
ideal storage device for large reference files 
such as training manuals, glossaries, 
encyclopedias, etc., computer-generated 
graphics, and digital audio. The large capacity 
frees the Instructional designer to be more 
creative in the use of varied instructional 
strategies and materials. The CD-ROM 
development is no more difficult than the 
development of any other CBT program and is 
priced to make more complex learning 
environments affordable. 

Selection of a CD-ROM Delivery System 

In the last five years the cost of a CD-ROM 
delivery system has substantially decreased, so 
that it is now within the reach of most training 
organizations. There are several 
considerations when estimating the cost of the 
delivery system. The selection of the delivery 
platform should be based on cost, availability, 
and performance considerations. A bare-bones 
80386 PC system, which costs less than $1,200, 
was determined to be the appropriate platform 
for the RPT-Trataer. To allow the program to 
run as smoothly as possible, it is Important to 
allow the CD-ROM to transfer data at Its 
maximum capacity. The faster the data can be 
transferred from the MS-DOS buffers to the 
memory, the faster the data can be transferred 
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from the CD-ROM. When a machine less 
capable than the 80386 was used, the graphics 
and screen changes were jerky and 
disconnected. 

At this time full motion video is not available 
on the CD-ROM; therefore, if the CBT is not a 
text only package, it is Important to have the 
best color graphics display. With the 
cost/performance ratio between Video 
Graphics Array (VGA) and older color 
standards comparable, the use of VGA, which 
allows the use of both graphics and scanned 
Images, is recommended. 

To take advantage of the digital storage ability 
of the CD-ROM, digital audio should be used. A 
digital audio system includes the cost of the 
audio board and the speakers. The audio board 
which we used was about $400. Other boards 
and playback systems are advertised for less. 
Thus, the cost of digital audio boards is 
decreasing. 

The selection of a CD-ROM player is dependent 
upon the system used. At this time, Apple, 
Commodore, and IBM. have different platform 
standards. If the delivery system is IBM 
compatible based, the CD-ROM player selected 
should conform to the ISO 9960 standard 
which was adopted by the International 
Standards Organization (ISO) to define a CD- 
ROM directory structure. The actual cost of the 
player is between $400 and $800. Thus, if the 
CD-ROM player is added to the basic delivery 
system of a 80386 PC with a VGA monitor, 
mouse and audio board, the total cost ranges 
from $3,000 to $4,000. The addition of a CD- 
ROM player has minimal impact on the 
delivery cost. 

Selection of a CD-ROM Development System 

Of course, the selection of a development 
system depends on the delivery system selected. 
The development system specifications are 
those of the delivery system with the addition 
of a read/write medium. Because the 
development of a training package is often an 
iterative process incorporating many changes, 
the development system needs to easily 
accommodate the changes. Because CD-ROM is 
a read-only device, multiple masterings during 
CBT development would be prohibitively 
expensive (approximately $1,500 a master). 
Therefore, hardware with read/write 
capabilities which mimic the CD-ROM is 
required. There are several different types of 
hardware devices available, including 
removable hard disk, fixed hard disk and 
magneto-optical removal disk. Listed below is 



the cost per megabyte (Dataware, Inc., 1989): 

removable hard disk $15 

fixed hard disk $10 

magneto-optical removal disk $10 

Since the costs are virtually the same, the 
decision must be based on other 
considerations. The performance of the 
development system should mimic the 
performance of the delivery system as closely 
as possible. 

Performance Considerations 

When selecting a development system for CBT 
for CD-ROM delivery, the performance of the 
CD-ROM needs to be considered. The slower 
access time and the larger capacity are the two 
conditions which are unique for the CD-ROM. 

Stew Access Time 

The performance characteristics of the 
training package are defined in terms of the 
CD-ROM drive performance and the 
operational characteristics of the software 
driver interfacing to the CD-ROM drive. 

The CD-ROM drive performance 
characteristics Include the data transfer rates 
and access time. The data transfer rates of the 
CD-ROM are 150 KB/s sustained and 600 KB/s 
burst. Access time Js 0.8 seconds for a full 
stroke seek. Seek time Is the amount of time 
necessary to move the mirror of the laser beam 
to the correct position on the CD-ROM. An 
average stroke is one -third of a full stroke, thus 
requiring 0.5 seconds. The typical rotational 
speeds of CD-ROM are 530 rpm at the 
innermost track with a constant linear 
velocity of 1.4 m/s and 200 rpm at the outmost 
track with a constant linear velocity of 1.2 m/s 
(Sony Corporation, 1988). 

In comparison, a typical hard disk has an 
access time of about 0.015 seconds, with a 
maximum track-to-track access time of 0.004 
seconds. The data transfer rate from the 
buffers is up to 6 MB/s (Electronic Engineering 
Times, 1990). These rates are considerably 
faster than the CD-ROM. Thus, the hard disk is 
not a good simulator of the completed CD-ROM 
product. 

With an average seek time of 0.0667 seconds 
(Relax Technology, 1988), the performance of a 
magneto-optical disk is closer to that of the 
CD-ROM than is the hard disk. The data 
transfer rate (about 690 KB/s) of the magneto- 
optical disk is faster than the CD-ROM. 
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Other Considerations 

In addition to the performance of the hard disk 
and the magneto-optical disk, there are other 
considerations when selecting a CD-ROM 
simulator. The capacity of the magneto-optical 
disk is comparable to the CD-ROM. The 
magneto-optical disk has two sides; each side 
has half the capacity of the CD-ROM. Hard 
discs of comparable capacity are available for 
about $3,000. 

Another Important consideration is the media 
which is sent to the CD-ROM mastering 
company. The CD-ROM mastering company 
requires the input media to be a tape cartridge, 
a magneto-optical disk, a Macintosh 
compatible hard disk, or a CD-ROM. If a hard 
disk is used, the data would have to be 
transferred to magnetic tape before mastering a 
CD-ROM. This step incurs an added expense for 
the magnetic tape equipment. If a removable 
magneto-optical disk is used, the cartridge can 
be sent to the mastering company, eliminating 
transferring the system to magnetic tape. As a 
point of comparison, when we compared the 
cost of the magneto-optical disk with the cost 
of magnetic tape equipment, the cost was 
$5,000 versus $30,000. At $5,000 for the 
magneto-optical disk system, the simulation 
CD-ROM delivery is affordable. 

Limitations of the Magneto-optical Disk 

We used a magneto-optical disk drive to 
simulate the CD-ROM. While In many ways the 
magneto-optical disk drive was an excellent 
mimic of the CD-ROM, it did have some 
limitations. The primary limitation was the 
number of allowable DOS buffers. The 
magneto-optical disk drive Increases the size of 
each buffer from 512 bytes to 8192 bytes. If the 
program requires a large number of buffers, 
there may not be enough memory left to load 
and execute the program. The RPT-Tralner did 
not require many buffers; we therefore, reduced 
the number of buffers in the conflg.sys file to 
five. 

Another limitation is the inability to use drive 
assignment statements. The CD-ROM driver 
does not allow the use of the DOS "subst" and 
"assign" commands. Therefore, it is necessary 
to use the magneto-optical disk drive in a 
single partition mode. Another constraint is, 
of course, the magneto-optical disk drive must 
have the same drive letter as the CD-ROM in the 
delivery system. 

Conclusion 

The CD-ROM is a viable alternative for 
multimedia CBT delivery, its large capacity 



allows the user to incorporate scanned images, 
large computer-generated graphics, and digital 
audio in a large CBT program. However, 
because it is a read-only device, there must be a 
separate delivery and development system. To 
allow testing, the development system must 
emulate the CD-ROM as closely as possible and 
yet be easy to use. The hard disk simulation of 
the CD-ROM is not recommended for two 
reasons. First, the performance 
characteristics of the hard disk and the CD- 
ROM are significantly different. The CD-ROM 
is much slower than the hard disk. This makes 
the package during development appear very 
different than the delivery package. Second, 
the data must be transferred to a tape cartridge 
before mastering. Because the magneto-optical 
disk drive and the CD-ROM have similar 
performance characteristics and storage 
capacity, the magneto-optical disk a good 
simulator of the CD-ROM. Furthermore, a tape 
does not have to be produced before the 
mastering of the CD-ROM. 
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COGNITIVE OUTCOMES OP BUILDING EXPERT SYSTEMS 



David H. Jonassen, University of Colorado 



Abstract 

Researchers and businesses are Investigating 
the effects of having students and employees 
construct expert system rule bases. What is 
important are the cognitive effects of the 
construction process more than the resulting 
advisors. Expert systems can be thought of as 
cognitive tools engaging learners in cognitive 
and metacognltive learning outcomes. This 
project proposes to investigate the changes in 
critical thinking skills, cognitive learning 
strategies, and metacognltive skills that result 
from employees and students constructing rule 
bases. Changes In individuals' knowledge 
structures will also be investigated. 

Background 

Building expert system knowledge bases Is a 
generative and constructlvistlc process that 
facilitates the acquisition of conceptual as well 
as procedural knowledge. The deeper 
understanding of the subject matter that 
results from the process represents a cognitive 
learning strategy. Cognitive learning strategies 
necessarily engage learners In higher order 
thinking. Just how high is the order of 
thinking that results from building knowledge 
bases? How do learners or employees 
intellectually benefit from assuming the role of 
knowledge engineer builders of expert system 
knowledge bases? Trollip and his colleagues 
(In press) believe that the development of expert 
systems results in deeper understanding 
because they provide an intellectual 
environment that 1) demands the refinement 
of domain knowledge, 2) supports probltm 
solving, and 3) monitors the acquisition of 
knowledge. Several constructs from learning 
psychology support their belief. The first 
requirement, the refinement of domain 
knowledge, Is met because building expert 
systems requires generative processing of 
Information. According to the generative 
hypothesis (Wlttrock. 1974). information 
becomes meaningful to the individual insofar 
as it is related to prior knowledge. Building 
expert systems requires the knowledge 
engineer to expose the prior knowledge of the 
expert. This entails Identifying declarative 
knowledge (facts and concepts), structural 
knowledge, the knowledge of the 
Interrelationships of Ideas in memory 
(Jonassen, Belssner, Jost, Kenny & Yacci, In 



press), and procedural knowledge (how to apply 
the former). Building experts systems, we 
believe, entails converting existing declarative 
knowledge into procedural knowledge and 
applying it. Finally, Trollip et al believe that 
environments should provide a mechanism for 
monitoring their own knowledge. This entails 
metacognltive awareness of their knowledge, 
which Is the highest order of Intellectual 
processing (Flavell et al, 1977). 

Clearly, building expert systems requires 
learners to synthesize knowledge by making 
explicit their own knowledge structures or the 
knowledge structures of experts. The 
improvement of retention, transfer and 
problem solving ability that results has been 
verified anecdotally by Trollip and others. 
This project seeks to provide empirical support 
for the hypothesis that building expert systems 
engages and Improves cognitive and 
metacognltive processing. 

This project seeks to analyze the Information 
processing requirements of building expert 
system rule bases. An analysis of information 
processing requirements (Jonassen and 
Grabinger, in press) showed that building 
expert systems engages individuals in a variety 
of higher order thinking processing, Including 
a number of cognitive learning strategies (see 
Table 1). 

This study proposes to collect evidence of 
cognitive and metacognltive changes in 
individuals as a result of building expert 
system rule bases that reflect personal 
knowledge of content and/or processes. 
Changes in critical thinking, cognitive 
controls, and cognitive structure of students 
and employees engaged in building expert 
systems will be assessed. 

Companies such as DuPont are training 
employees how to build rule bases that describe 
the decision making in their operations. They 
are finding that by providing operating 
departments with small rule- based shells, they 
are engaging employees in a very deep analysis 
of their operations. They are finding that the 
resulting rule bases are of marginal utility 
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Table 1 

Levels of Processing In Building Expert Systems 


Step 


Gagne 


Bloom 


Tessmer& Jonassen 


Brezin 


1. Identifying problem 


Higher rule 


Anal ysls 
Evaluation 




Testing 
Judgi ng 


2. Limiting problem 


Concepts 
Rules 


Analysis 


Organizi ng 


El a bo rati ng 
Revising 

Testi no 


3. Specifying solutions 


Rule 

Higher rule 


Analysis 

(Synthesis) 

(Evaluation) 


(Elaboration) 


Revising 
Testi ng 


4. Specifying attributes 


Concepts 
Rules 


Analysis 
(Evaluation) 


Organizi ng 


El a bo rati ng 

Relating 

Revising 


5. Solution matrix 


(Concepts) 
Rules 


Synthesis 


Elaboration 


Revising 


6. Generating rules 


Rule 

Higher rule 


Synthesis 


Elaboration 
Sequencing 


Revisi ng 
Testing 


Note: Parentheses indicate a limited role. 



(perhaps as a Job aid for new employees), but 
the effects of the analysis of their operations 
necessary for representing them in a rule base 
are quite often dramatic. Improved efficiency 
and effectiveness often results from this 
analysis process. So, in addition to 
functioning as a cognitive tool, rule-based 
expert systems may also function as a systems 
analysis tool. 

Approach 

The first step will be to identify measures of 
thinking and systems outcomes. A review of 
the literature to identify tools or techniques 
that evaluate the effects of systems analysis 
will be conducted. A review of measures of 
higher order thinking will also be conducted. 



University students from the University of 
Colorado and employees at Coors will be 
pretested with measures of critical thinking 
skills, metacognitive skills, cognitive controls, 
and cognitive structure. Critical thinking 
skills will be assessed by the use of tests such as 
the Watson-Glaser Test of Critical Thinking, 
Cognitive controls, such as Cognitive 
Complexity/Simplicity, Field Independence/ 
Dependence, and Necessary Arithmetic 
Operations. Effects on knowledge structures 
will be established by pre- and post-test 
comparisons of semantic nets of subjects, 
generated by Learning Tool, will also be 
assessed. Cognitive structures of each 
participant will be assessed using a cognitive 
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mapping technique and backed up with word 
associations, the benchmark technique for 
assessing cognitive structure. The purpose of 
this measure is to assess the effects of the 
knowledge engineering process on knowledge 
structures/The first is a public domain shell 
developed by Tony Starfield at the University 
of Minnesota. It is simple to use ani can be 
trained readily. More advanced development 
will depend upon a more sophisticated shell, 
probably VP Expert. 

Procedure 

The Instruction and construction of rule bases 
will be accomplished on MS-DOS machines. 
Laboratories at the University of Colorado will 
be used for this purpose. Coors has agreed to 
participate by allowing their employees to 
construct knowledge bases. 

Following pretesting, students and employees 
will be trained in the university or at their 
place of work in using simple, MS-DOS expert 
system shells. Some of these are public 
domain, while others are quite Inexpensive. 
Students will be required to develop content- 
oriented rule bases while employees will be 
requested to simulate the operations or 
decision making in their organization in the 
knowledgebases. Each of the knowledgebases 
will be validated, using subjects from the target 
audience. 

After producing one to three knowledge bases, 
learners and employees will be posttested with 
the above instruments. Evaluations of the 
systems In the company using the instruments 
or techniques identified in the first part of the 
study. 

Potential for Broad-based Technology Transfer 

Expert systems represent a useful cognitive tool 
and systems analysis tool, a computer-based 
tool that engages and supports generalizable 
cognitive processing (Kommers, Jonassen & 
Mayes, in press). As a generalizable tool, expert 
system construction can be used as an 
instructional activity in virtually every 
content domain and in most every operation in 
any business. Although corporations do not 
focus on intellectual self-aggrandizement, the 
effects of an enlightened work force cannot be 
underestimated. 

The benefits of knowledge engineering extend 
beyond the development of generalizable 
learning skills. Building expert systems can 
also function as a systems analysis tool.for 
evaluating operations. This project proposes to 
identify and evaluate those effects. 
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Abstract 

Issues related to the design, development, and 
testing of a contextuallzed computer 
environment for Spanish language acquisition 
are discussed In relation to the role of computer 
in school settings and the need to embody 
cognitive learning theories In computer-based 
instructional systems. 

Context is an important component of any 
learning activity. Research has shown that 
initial pattern recognition (see Anderson, 
1985), subsequent retrieval from long-term 
memory (Tulving & Thompson, 1973), and 
comprehension (Carpenter & Daneman, 1981) 
are affected by context, and that problem 
solving is also dependent on the context of the 
situation (Maier & Janzen, 1968). These 
findings have added to the growing speculation 
that the context of learning and problem 
solving situations influences thought 
processes, and that instruction that Is situated 
in meaningful settings is more effective 
(Brown. Collins, & Duguid, 1989; Greeno, 1989). 

In the field of foreign language instruction, a 
related shift in theoretical orientation has 
recently taken place. Rather than memorizing 
dialogues and practicing pattern drills, foreign 
language educators now believe that successful 
acquisition of a second language results from 
experiences which support natural language 
learning strategies such as language play (Peck, 
1980), semantic processing (Hatch, 1978), and 
delayed oral practice (Postovsky, 1977). 
Research suggests that such strategies are best 
implemented in learning environments that 
are rich in context (Wong-Fillmore, 1989). 

Parallel to the development of new curricular 
and instructional approaches in foreign 
language education, computer technology has 
sufficiently advanced to a point where the 
design of effective contextuallzed computer 
environments (CCE) is possible. These 
environments utilize pictures, drawings, sound 
and text to illustrate key ideas, events, or 
stories which engage learners in listening for 
information and demonstrating 
understanding. In the following pages, we 



present the results of extensive design, 
development and testing activities related to a 
CCE for Spanish language acquisition with the 
hope that others interested in designing 
computer learning environments will benefit 
from our descriptions. 

Design Assumptions and Features 

The basic assumption made In the design of our 
CCE is that communicative competence in a 
second language is not learned, but acquired. 
While controversial, this distinction as 
proposed by Krashen (1982), is based on the 
notion that acquisition of language abilities 
involves subconscious processes associated 
with the language "mental organ" (Chomsky, 
1965). Learning a second language, on the other 
hand, requires conscious processes related to 
rule learning and correction of language 
performance errors (Krashen, 1985). The 
processes of acquisition depend on 
"comprehensible input" as an essential 
environmental ingredient (Krashen, 1982). 
Messages containing language forms Just 
beyond the student's present level of 
competence, when presented in a familiar 
context, provide sufficient amounts of 
comprehensible input to foster language 
acquisition. Such conditions are difficult to 
provide in a traditional classroom since, in 
most cases, the teacher is the only speaker of 
the target language, in such instances, 
researchers have recommended that a 
"language-rich environment" be provided in 
order to allow children to interact in a variety 
of contexts and functions (Wong-Fillmore, 
1989). 

A CCE may serve as a viable alternative to 
provide a language-rich environment for 
elementary school settings. Essential factors 
for second language acquisition, including 
one's knowledge of the linguistic code, 
knowledge of the world, and knowledge of 
discourse structure (Omaggio. 1986), can be 
easily activated by one resource (the computer). 
Contextuallzed exercises (Omaggio, 1986), 
which "anchor" instruction in meaningful 
situations (Bransford, Sherwood, & 
Hasselbring, in press) encourage students to 
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attain goals by dealing with problems that 
require recognition of features in given 
problem situations. Natural learning 
strategies such as language play, use of 
formulaic expressions, and considerable 
amounts of listening to messages in the target 
language should be provided in a CCE in order 
to enhance the acquisition process. The 
problem we encountered during the design 
process, however, is that many of the 
procedures and techniques prescribed by 
traditional instructional design models are not 
applicable to the design of a CCE. This problem 
is mentioned here only in passing, since it Is 
beyond the scope of this paper to address the 
deficiencies in current design models (See 
Nelson &Orey. 1991). 

With "Salamanca", the CCE described below, 
we have attempted to incorporate several of the 
design features just mentioned. Our main goal 
is to provide an electronic setting for receiving 
comprehensible input in spoken and written 
forms through a simulated visit to the Spanish 
city Salamanca. During the visits, the user 
meets two Spanish children, and engages In a 
variety of activities that require listening to 
recordings of spoken Spanish, reading 
captions or Instructions In the language, and 
responding or selecting within simulated 
precommunicative practice activities. In 
addition several instructional activities are 
Implemented in the CCE. including cloze 
activities (sentence completion), 
precommunicative practice (authentic 
information about the culture derived from 
written and spoken communication), and 
contextualized practice (listening and 
demonstrating understanding). 

The CCE opens with a "tour" of the city, which 
Includes numerous opportunities for obtaining 
cultural information by clicking on objects 
within the city (buildings, bridges, a coat of 
arms, etc.). As the user continues on Uie tour, 
Paloma, the protagonist in the electronic 
environment is introduced, and presents a 
series of descriptions, in Spanish, that provide 
general information regarding the operation 
and content of the different areas of the 
software (See Figure 1). Paloma also introduces 
the user to several settings in Salamanca, 
including avenida Alemania, Anaya Palace, 
and Plaza Mayor. The p'aza provides the 
setting for two activities, accompanying 
Paloma on a shopping errand, and ordering a 
snack at an outdoor cafe. In both activities, the 
students hear authentic dialogues designed to 
provide both comprehensible input and a series 
of formulaic expressions needed for 



communication in certain settings. The user 
also learns how to call Paloma on the phone 
and visits the schools, where a dictionary, 
maps, a grammar book and several 
Instructional lessons can be accessed. 

During the tour the user is also introduced to 
Manolo, Paloma's cousin, and is invited to his 
house. If the student chooses to visit Manolo. a 
sequence unfolds where the correct address 
must be found in a address book and the correct 
doorbell must be rung. Unfortunately, the user 
learns from Manolo's mother that he is at the 
park. If the user chooses to go there, the games 
"Hide and Seek" end "I spy" can be played (See 
Figure 2). Similar activities are provided if the 
user visits Paloma's apartment, where "Veo 
veo" (a guessing game) can be played, songs can 
be heard on the phonograph, or the learner can 
read Spanish comics. 

After completing the initial "tour", the user is 
free to explore any component of the software. 
Both Paloma's and Manolo's apartments can 
be visited, or the user can call them on the 
phone (See Fi„ ire 3). A dictionary of Spanish 
words and a "repeat" feature for some of the 
recorded dialogue is available at any time. The 
user also has the opportunity to visit the 
school, where several authentic activities are 
provided, including a recorded and illustrated 
"lesson" describing wine making, a language 
arts lesson featuring sentence completion and 
letter writing tasks, and a "cultural safari" 
where users can respond to various situations 
requiring cultural knowledge (See Figure 4). 

The design assumptions made and the 
subsequent solutions implemented reflect the 
current theoretical orientation of foreign 
language education and cognitive learning 
theories. The emphasis on exploration, the 
high degree of context, and the authentic 
dialogues and communicative practice all 
contribute to a learning environment that 
differs significantly from the traditional 
classroom. It remains to be seen whether these 
differences contribute positively to learning a 
foreign language. The requirements of the CCE 
design had an impact not only on the 
instructional design process described above, 
but also required different approaches to 
development and evaluation than are 
normally prescribed by instructional design 
models. Some of these variations are described 
In more detail in the following pages. 
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Figure 1. Paloma greets new visitors to Salamanco 



The Development Process 

Salamanca was developed by a three-person 
team over the course of three months using 
HyperCard on a Macintosh SE computer. The 
team was led by the designer who was a 
Spanish teacher, and who also produced line 
drawings used as graphics in the software. The 
second team member was a programmer who 
constructed the stacks and wrote the code 
necessary for the various components of the 
software. The third team member assisted with 
the programming, and was responsible for 
recording and installing the sound resources 
Into the stacks. 

A rapid-prototyping approach (Tripp & 
Bichelmeyer, 1990) was utilized throughout the 
project. A storyboard prepared by the designer 
guided the development process. Initial stacks 
were developed quickly as shells, and then 
modified when graphics and sounds were 
completed. Several development tools, 
Including a scanner for creating graphic 
Images, sound digitizing equipment and 
software (Farallon Computing, 1989), and a 
resource utility (Mailer, 1988) helped to speed 
the development process. Original graphics 
created by the designer were scanned and 
Imported into the stacks, and In some cases, 
enhanced with commercial "clip art" Images. 
The software was also Instrumented with data 
collection scripts so that Individual users could 
be tracked. This data, Including the sequence 
of cards visited and the amount of time spent 
on each card, was stored In a disk file after each 
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session. 

The recording of sounds for the stacks was 
completed In just a few hours, but several more 
hours were needed to edit the sounds (removing 
unwanted silence and smoothing distortion) 
and compact the sound files. In fact, the 
extensive size of sound resources (even when 
recorded at a "compressed" sampling rate of 4K) 
caused the greatest problems during 
development. The initial design specifications 
called for four stacks (the Tour, Paloma's 
house, Manolo's house, and the school), but 
ultimately the program had to be split into ten 
stacks that totaled 4539K of disk storage space 
because we wanted no individual stack to be 
larger than 800K. Nearly 3000K of this total 
was taken up by sound resources. 

Results of Testing 

The CCE was pilot tested with thirteen 5th and 
6th grade students at a "magnet" school In a 
large mid western city during the Fall of 1990. 
All students were familiar with computers, 
although not the Macintosh, because their 
regular classroom instruction incorporated 
some drill and practice software for grammar 
and vocabulary activities. Students worked in 
groups of three at the computers for 14 sessions 
over the course of two months. 5th graders 
spent 35 minutes working with the CCE at each 
session; 6th graders spent 45 minutes. The 
results reported below are part of a larger study 
which Implemented a qualitative approach to 
data collection in order to ascertain the Impact 
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Figure 2. Playing In the park with Manolo. 





Figure 4. The schoolhouse. 



of the CCE on the classroom setting. 
Ethnographic Interviews, participant 
observation, oral proficiency interviews, 
writing samples, and data collected by the 
software during student Interactions were used 
to describe details regarding student 
Initiations and strategies in using the software. 

Nature of interactions. In the computer 
sessions, several types of initiations made 
during peer Interactions were observed 
classified and recorded. Table 1 provides a 
summary of the findings. The largest number 
of initiations (76%) were cooperative in nature. 
As the students became more familiar with the 
software and gained more experience working 
together in their groups, initiations seemed to 
Include more cooperative overtures and greater 
direction In regard to the activity in which they 
were Involved. 

It was observed that computer-generated 
sequences involved several distinct 
Interactions that occurred almost 
simultaneously. Instead of extended 
Interactional sets, the computer-generated 
sequences often spawned multiple extended 
Interactional sets. Responding successfully to 
the computer initiations required a 
cooperative effort. Even when the students 
established their roles within their groups and 
exerted a cooperative effort, multiple 
interactions continued. In addition, 
responding appropriately depended on the 
ability to make sense of the computer 
initiations. 



A number of student explorations during the 
sessions with "Salamanca" revealed the nature 
of learner strategies used to regulate learning 
with the CCE. In the tour, the students were 
Introduced to the notion of exploring the 
environment by clicking on objects. The first 
card In the stack exhorted the students to 
explore, and a few students found the hidden 
fields describing (in Spanish) the objects in the 
environment. Most students closed the fields 
without reading them; two students scanned 
the first field for familiar words before closing 
the field. Several student Interactions were 
negotiations about what object should be 
clicked. All of the students seemed to be 
intrigued by the ability to initiate change 
among the objects on the screen. Sometimes 
these changes Imitated communicative 
changes. For example, the icon labeled "Adios" 
was used by students to exit the program. 
Several students were observed saying 
"goodbye" when they clicked the icon. 

Many other examples of playful Interactions 
were noted. Students were intrigued by the 
visual effects used for card transitions, and the 
data files indicated that the students often 
moved quickly through certain card sequences 
in order to see the visual effects. In particular, 
one sequence was designed to be an animation 
that accompanied a song on cassette tape. 
Several students initiated this sequence 
without the tape; one student was observed 
completing the sequence while humming the 
song. During one observation the researcher 




watched a group of students click on the words 
in the dictionary several times in succession. 
They laughed and looked at one another before 
repeating the action again. They appeared to be 
creating an echo effect. On another occasion, 
the sixth grade group rang all of the doorbells 
at Manolo's apartment building. They rang 
some of the doorbells more than once. In 
general, tine students were attracted by the 
computer's flash, but they also were drawn to 
substantive exchanges with the softwr c In 
exploring the CCE, the students sought to 
manipulate the environment to create 
communicative exchanges. 

In exploring the target language messages in 
the environment, the students appeared to 
develop some resourcing strategies. 
Observations and data from the compute: files 
noted many uses of the dictionary. They would 
often interrupt an activity in order to explore 
the dictionary before returning to the activity. 
In fact, the students' use of resourcing alerted 
the researchers to items that needed to be added 
to the dictionary. Other student explorations 
revealed the need for a "repeat dialogue" button 
on many car ds. In the initial design, many of 
the recorded messages were played when the 
card was opened. Since many students were 
observed nipping between cards to repeat the 
messages, the "repeat dialogue" buttons were 
added. 

Constructing lessons with "Salamanca". The 
nature of the lessons that were constructed by 
the students through their interactions with 
"Salamanca" were reflected by the information 
collected in the data files. Table 2 summarizes 
the recorded student-directed interactions with 
"Salamanca" that occurred during the 
sessions. 

Several types of exploration were documented. 
For example, during one session the sixth grade 
students flipped through several pages of a 
story, pausing cn cards where there were 
pictures. They also flipped forward and 
backward through the story sequence, never 
staying on any card more than two seconds. 
During another session, students spent over 
two minutes looking at a language arts writing 
assignment, but they did not write anything. 

Subsequent opportunities to explore and direct 
their own interactions with the software 
revealed some patterns In student constructed 
lessons with "Salamanca" First, students 
continued to seek opportunities to listen to 
recorded Spanish. Although the students had 
written down the characters' phone numbers 
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and addresses, they continued to try to follow 
the tour sequence by listening and accessing 
the information that they needed from the oral 
and written messages in the software. Some of 
the lessons constructed by the students seemed 
to be used to attain the student agenda of 
"thinking and remembering". For example, on 
several occasions the students selected to 
participate in the lesson on wild animals in 
Spam. In early interactions, they appeared to 
Just be browsing, but during subsequent 
interactions, the students began to respond to 
the lesson activities. These interactions 
seemed to provide the students with 
opportunities to recognize and remember 
familiar vocabulary. 

The students engaged in practice activities 
which differed from the formal practice 
required of them in the traditional classroom. 
According to the students Interviewed, this 
practice entailed: "getting the general idea of 
what you need to do", "getting the clues for the 
entry words... to get in the passageways", 
"getting where you want and finding what you 
want to find", "answering in Spanish", and 
"grasping a little more Spanish because you're 
actually seeing it and seeing how it applies." 
The students practiced making sense of the 
simulated target culture environment by 
creating conditions for their learning. 

The students also sought opportunities to 
respond in the target language. Students 
generally responded and initiated by using 
familiar set phrases. For example, the fourth 
card of the tour contains the picture of a girl 
and a recorded message says: "Hi friends. 
Welcome to Salamanca." A student was 
observed saying "Hi! How are you?" in 
response to this recorded message. In addition, 
the students' experiences with "Salamanca" 
seemed to broaden their concept of second 
language learning. The following excerpt sums 
up how listening to recorded Spanish took on 
new values: 

Researcher: How would you describe the 
talking on "Salamanca"? 

Christina: Because they have questions 
like... something like <what a pity> or 
something. They wouldn't say that on the 
"Spanish Bullfight" They would Just 
command you to do what to do. Like type this 
answer, do this, do that. It's not like that with 
"Salamanca". "Salamanca" is... it's more Just 
to get the general idea of what you need to do. 
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TYPE 



COUNT 



EXAMPLE 



Peer Assist 



10 



12 



Navigating 
Peer Evaluating 



Request Info, 
from peer 



Directing 
Strategy 

Request a turn 

Share 

Competitive 
Compete 



Usurp 2 
Squatter's Rights 1 



Policing l 
Directing 2 
Peer Evaluating 4 
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10 

3 
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Ana looks down at the keyboard. Christina points to the 
return key. Ana nods her head up and down. 
"Favor...spell It like favor*. F-A-V-O-R." 

"Where do you want to go?" 
"What should we click on?" 

"You forgot 'favor*". 

"Yeah, inarca' that means to call." 



"What do we put?" 
"What's that?" 

"Go to 'Cuenta', or whatever you call It." 

"Put a comma. You know, like when you write your last 
name first." 

"Let me..." 

"Do you want to go first?" 



Oscar clicks on the window of a house instead of 
following the suggestions of his peers. 

Concha reaches across the keyboard while Jose is typing 
and hits the delete key several times. 

The first students to enter the laboratory scramble for the 

chairs around the computer. 

"Do you want to control the mouse?" 
"Go.. .Go forward." 
"Don't do that." 



Table 1. Number and types of initiations among peers while using "Salamanca" 
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Session Stasia. Total 

Accessed Cards 

la Tour 

Paloma-House 
Cuento 

Manolo-House 183 

Manolo-Phone 

Paloma-Phone 

Dictionary 

Lessons 

lb Tour 

Paloma-House 

Dictionary 
Manolo-House 89 
Manolo-Phone 
Paloma-Phone 

2a Tour 

Manolo-House 
Paloma-House 
Dictionary 106 
Verbos 

Manolo-Phone 
Lessons 

3a Tour 

Paloma-House 
Dictionary 93 
Lessons 

3b Manolo-House 
Dictionary 
Manolo-Phone 188 
Verbos 
Lessons 
Safari 

4a Paloma-Phone 
Dictionary 
Manolo-House 99 
Manolo-Phone 



4b Paloma-Phone 
Tour 

Dictionary 66 



Average Sees. Type of Exploration 



11 



19 



18 



19 



11 



21 



31 



browsing (games, lessons) 
flipping through (story) 
input (pastry shop) 

precommunlcative practice (phone-house) 
truncated interactions (game) 
play (doorbells) 
navigating (story) 



resourcing (dictionary) 

flipping through (comics) 
precommunlcative practice (phone-house) 



input (tour) 
navigating (external) 
resourcing (repeat dialogue) 
browsing (lessons) 
participating (lessons) 
play (doorbells) 
dueling (options) 

researcher directed activity (song) 
browsing 



following other group's activity with 
researcher 

flipping (story, comics) 
context (story) 



Input (pastry shop) 
resourcing (dictionary) 
context (comics) 

precommunlcative (house-phone) 
flipping (comics) 

navigating (external) 
resourcing (dictionary) 
Input (pastry shop) 



Manolo-House 



reading (comics) 
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Table 2, continued 



5a Manolo-House 

Dictionary 105 
Manolo-Phone 

5b Paloma-Phone 
Manolo-House 
Manolo-Phone 
Verbos 146 
Dictionary 
Tour 
Lessons 

6a Cuento 

Paloma-House 148 

Dictionary 

Tour 

6b Cuento 

Paloma-House 

Dictionary 263 

Lessons 

Verbos 

Safari 



15 



12 



13 



play (doorbells) 
dueling (options) 
flipping (comics) 



browsing (dictionary/lessons) 
navigating (school) 
flipping (verbs) 



flipping (story) 
resourcing (story) 



flipping (story) 
navigating (icons) 
resourcing (story) 
play/duel (school) 



Table 2. Student-directed explorations of "Salamanca". 



Conclusions 

In designing Salamanca, we attempted to 
address learning problems related to the 
classroom environment and the use of the 
target language in that environment. The 
activities Implemented In the software were 
derived from communicative contexts and 
natural language learning strategies. During 
development and testing, we have sought 
insights regarding the potential use of a CCE as 
a resource for obtaining comprehensible input 
and for practicing learning strategies that may 
facilitate the development of language 
proficiency in the Spanish classroom settings 
of our schools. Preliminary results are 
encouraging and have identified several issues 
for further Investigation. 

Whether exploratory environments like 
"Salamanca" should be structured to provide 
exposure to concepts appropriate to the 
intellectual development of learners or 
whether they should present simulated 
environments where students can expand their 
grasp of the task of learning a second language 
merits further research. It appears, however, 
that contextuallzed computer environments 
have value for learners as resources for 
accessing target language meaning. in 
addition, "Salamanca" appeared to expand the 
learners' concept of learning Spanish. Finally, 



the students' interactions with "Salamanca" 
seemed to encourage participatory and 
cooperative learning behaviors. 
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THE COACHING NETWORK: A MODEL FOR CONDUCTING AND MANAGING 

LEARNING 



Jennifer G. Smith MA, EG&G Rocky Fiats 
Eric Smith Ph.D„ University of Northern Colorado 



ABSTRACT 

The Coaching Network Is a mechanism for 
continually instructing and providing 
feedback to the learner during and after formal 
instruction. Six conditions necessary for the 
Implementation of a Coaching Network are 
discussed. Use of the Coaching Network leads 
to improved performance, Independent 
learning, improved skill /knowledge, and 
goal/objective setting. 

OBJECTIVES 

Two objectives have been developed to drive 
and evaluate this paper, 
•the reader will be able to state the goal and 
definition of the Coaching Network and 
•the reader will be able to describe the process, 
model, and results of the Coaching Network 
as It applies to Industry and academic 
settings. 

INTRODUCTION 

An instructor's goal is to improve the 
performance of the students. That, 
performance may be the ability to efficiently 
complete a task or the ability to seek out a 
solution to a problem. Unfortunately, 
Instruction occurs with no guarantee that the 
students will actually use what was learned. 
Therefore, additional methods must be utilized 
to ensure continual learning transfer. The 
Coaching Network has been developed from the 
analysis of athletic coaching strategies, 
instructional strategies and management 
techniques and Is utilized to provide continual 
Instruction and feedback to the learner. 

DEFINITION OF COACHING 

We define coaching as an individual/small 
group process of interaction among coach and 
team members with the expected outcome of 
independent performance (Rudd & Smith, 
1990). This does not mean that team members 
are expected Just to work alone. Instead, the 
goal Is for the team to be able to function 
individually and together in a self-directed 
manner, with or without explicit instruction 
from a supervisor (or coach). Looking at this 
idea in terms of a race, it is the ieam member 
who must accomplish the race goals, not the 
coach. But without the coach and other team 



members, the person running the race cannot 
achieve the goals. It is the same in any 
environment. 

THE COACHING NETWORK 

The Coaching Network Involves seven steps; 
listed below. 
1. Identify institutional philosophy, 

2. Determine personal philosophy, 

3. Create positive environment, 

4. Develop team goals and objectives, 

5. Develop personal goals and objectives 
related to team goals and objectives, 

6. Orchestrate individual and small group 
assignments related to tasks of the group 
and 

7. Evaluatlon of coaching process and 
progress toward all goals. 

In identifying the Institutional, philosophy, the 

coach must determine the institutional goals 
and attitudes toward goal attainment and 
employees. The institutional attitudes will 
have a constraining effect on Implementation 
options for coaching and team building. This 
means that the coach (manager) and team 
members must have a philosophy that is 
compatible with that of the company in order 
to be an effective team and to avoid 
unnecessary conflicts with management. 
Identification of the Institutional philosophy 
is accomplished through a combination of 
talking with superiors and examining existing 
documents (such as annual reports, memos and 
other internal communications). 
Consistencies and inconsistencies are 
identified and compared with the coach's own 
philosophy. 

The identification of personal philosophy and 

its "fit" with the institutional philosophy is 
critical for the success of the Coaching 
Network. If the personal philosophy Is in 
conflict with the institutional philosophy, 
development of effective teams will be 
hampered at best. However, there is more to 
personal coaching philosophy than simply 
matching the Institution. The coach must 
Identify the personal coaching philosophy 
(management approach) that will be used to 
guide and motivate the team members. 
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Personal goals and how those goals relate to 
the members of the team and the institution 
must be identified. The relationship among the 
coach and team members must be determined 
(e.g. formal, informal). Linked to this 
relationship is the identification of the 
leadership style used by the 

coach. In other words, the style of the 
environment (motivation, collaboration, 
individual and group focus on goals) is 
determined by the personal philosophy. 

Creation and maintenance of a positive 
environment includes communication paths 
(both formal and informal), information 
availability, appropriate support (both 
individual and small group), availability of 
appropriate resources (for example, key people, 
hardware, software, course materials, library 



materials), and feedback mechanisms 
(including cheerleader and mentoring). In 
addition, positive attitudes and collaborative 
effort must be encouraged. As part of the 
environment, instruction must be provided. 
This instruction is both formal, a classroom, 
and informal, on the Job training. Finally, 
care must be taken to build self-esteem on the 
part of each team member. This can be done 
through both formal and informal means. A 
positive environment does not mean a lack of 
discipline or consequences for Inappropriate 
behaviors of team members. However, 
consequences are made clear prior to problems 
arising and are fairly and impartially 
administered. 



Coaching Chain 



Coaching 

Information 

Support 

Resources 



An individual/small group process of interaction 
among coach and team members with the expected 
outcome of independent performance. 

Knowledge and skill transger 

Positive lndicidual/small group interaction 

Locating and utilizing key players tools 




Figure 1 The Coaching Cham 
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In order to function, a team must have team 
goals. These goals may be defined by the 
administration (institution) or the coach, but 
they are most effective if the team members 
have a part in the decision. This leads to "buy- 
in" and motivation to attain the goals. The 
development of team goals can be 
accomplished by presenting the goals and 
expectations of the company to the team, 
inviting members to discuss the goals and 
identify their fit. Next, the team decides how to 
attain the goals. In this process, the team 
identifies the short and long term goals. Of 
greatest importance, however, is the team 
working together to identify the goals and 
reaching a consensus. 

Once team goals are identified, individual 
goals/objectives supporting must be developed. 
This requires a one-on-one approach, with the 
coach working individually with each team 
member, setting goals that are achievable, 
challenging, and At with the team goal. In this 
way, each member's relationship with every 
other member is defined, much like positions 
in basketball. Having defined roles, allows 
coordination of team efforts. 

Orchestration of individual and small group 
assignments is a management problem. The 
coach must coordinate the assignments, 
whether they are classroom assignments or 
work tasks, so that they are focused on 
achieving the team (and in so doing, 
individual) goals. Communication about 
progress and problems is critical in this part of 
the model. Communication provides 
opportunities for cheerleading and group 
problem solving. 

Evaluation of the coaching process and 
progress toward all goals/objectives is an on- 
going process. Results of the evaluation are 
used to modify team relationships, goals, 
tasks, and access to resources, support, and 
information to improve the success of the 
team. Success is measured in terms of: 
•attitudes 

•Increased interaction among team members 
and other teams 

•progress toward goals/objectives and 
•attainment of goals/objectives. 

The outcomes of the evaluation process 
include: 

•identification of individual needs, 
•identification of problems with the coaching 
process, and 

•identification of new goals. 



THE COACHING CHAIN 

The Coaching Chain model, Figure 1, 
represents the Interaction of each team 
member with information, support, and 
resources needed to improve performance and 
or knowledge on the Job. The support and 
resources needed include fellow team members; 
In effect each member becomes a peer coach. 
The coaching process also occurs from team 
member to subordinate and superordinate 
teams. As the process evolves, the additional 
links in the chain are formed. It Is the chain 
and its growth that underlies the success or 
failure of a coaching model for management. 



When this model is implemented in a setting, 
say a college course on computers in education, 
the instructor functions as the coach. During 
the first several class meetings, the ground 
rules for communication and the class goals 
are set. In individual meetings with the 
instructor, each 'jlass member identifies 
his/her fit with the team. This includes 
identifying both strengths and weaknesses 
each member brings to the class. The 
instructor then organizes the class into 
working groups whose focuses are on how each 
group will contribute to the class goals. 

At this point, the Instructor begins the process 
of instruction. That is, some direct "teacher- 
lead" instruction occurs. The purpose is to 
provide a minimum set of skills for each 
member such that the groups can begin to 
explore the information and skills needed to 
accomplish their tasks. As the exploration 
progresses, the Instructor becomes a facilitator 
or guide, providing suggestions and directions 
as needed. On occasion, some additional direct 
instructions occurs to fill in some details or 
skills that have not yet been learned, but are 
needed to move forward toward the goals. 

As the course continues, the individual class 
members begin to share individual expertise, 
contributing both to the accomplishment of the 
goals and to the general skill/knowledge level 
of all class members. By the end of the course, 
the goals are met and all of the students have 
increased their skills/ knowledge. In effect, 
they have become their own Instructors, with 
the course instructor acting as guide and 
mentor. 

Several results are gained from implementing 

this process: 
•Improve skill/knowledge mastery, 
•Improve Independent learning, goal setting, 
achievement, and performance, 
•Increased group cohesion leading to 
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Improved performance, and 
•Improved performance leads to Increased 
production. 
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THE IMPACT OP COMPUTER BASED LEARNING ON EXAMINATION 
PERFORMANCE IN MEDICAL BIOCHEMISTRY 



James Baggott, Ph.D. and Sharon E. Dennis, M.S., 
Hahnemann University 



Abstract 

An optional student-controlled computer- 
based learning (CBL) environment (lecture, 
review, self-testing and clinical problem 
modules) covering over 50% of a Medical 
Biochemistry course was made available to 
first year students. Users experienced 
improved performance on multiple choice 
examination questions. The percentage 
Improvement on questions for which CBL was 
used is predicted by the intensity of CBL use. 

Introduction 

In 1988 we posed the question, "What would 
happen if an entire medical school course were 
replaced by computer based learning?" 
Implicit in this question were several concerns 
that bear on basic science education in medical 
school, and on medical education in general. 
The question also made certain assumptions 
about the nature of the computer based 
learning that would replace the course. 

The educational concerns included the 
following: Medical schools typically have 
overcrowded curricula which are heavily 
scheduled, and in which students are expected 
to master a very large and growing volume of 
facts in a short amount of time. 
(Chandrasekharan, 1988) Preclinical courses 
are often taught in a block that is separated 
from clinical considerations, both in time 
(being presented during the first year and a half 
to two years) and in content (being taught by 
Ph.D.s who sometimes give little attention to 
the clinical interests of their students). 
(Rothstein, 1987) The teaching and learning 
processes are often examination driven, with 
the immediate focus being course 
examinations, and the long term focus being 
the National Board of Medical Examiners Part 
I Examination. Courses are usually taught by 
the entire staff of the department, but 
sometimes with little Integration among the 
faculty, except at the level of assigning specific 
topics to specific faculty members. 

It seemed to us that the kind of CBL program 
needed to address these concerns would be a 
complete, student controlled learning 
environment. It would contain four major 



elements (Dennis, 1989): (1) The complete 
material content of the course in detail, 
Including all graphs, tables, graphics 
(animated if appropriate) and behavioral 
objectives. (2) Review and drill materials. (3) 
Examination questions for self-evaluation. (4) 
Realistic applications of the basic information 
to clinical situations. The student would 
determine what activity he/she would engage 
in at any time, and the program would allow 
the student to move freely among types of 
learning within a topic as well as from topic to 
topic. 

We embarked upon the construction of such a 
program for a Medical Biochemistry course, 
using HyperCard and the Macintosh computer 
(Dennis, 1990). The project, the Integrated 
Biochemistry Learning Series (IBLS), Is now 
well over 50% complete (Table 1), and contains 
modules covering biochemistry topics 
occurring at all points In the course, from 
beginning to end. This Is an interim report on 
the effects of this type of CBL on standard 
multiple choice examination performance. 

Methods 

The experimental population consisted of 243 
medical and graduate students taking Medical 
Biochemistry at Hahnemann University In 
Fall of 1990. 

Students were Introduced during the course 
orientation to the availability of CBL. and they 
were told that IBLS could be used on the 12 
Macintosh computers In the Microcomputer 
Learning Laboratory of the University Library 
during all regular library hours. Those with 
appropriate computer hardware were also 
offered, at no charge, a personal copy of IBLS. 
On the second day of the course the software 
was demonstrated during a regular class 
period, using a video projector in a lecture hall. 
The demonstration described the content and 
scope of the program, explained strategies for 
integrating its elements into a coherent study 
plan, and taught the specific techniques and 
conventions of the software, such as clicking 
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Table 1 



Portions of the Integrated Biochemistry Learning Series 
Complete at the Time of This Study 



"X' denotes a module that was complete for the Indicated topic. 



Twr * 1WF ' 



X 



Amino Acids 
Bioenergetlcs 
Carbohydrates 

Enzymes X 

Hemoglobin and Gas Trans. X 
Lipids 

Macromolecules X 

Metabolic Interrelations X 

Nutrition X 
Hormones from Amino Acids X 

pH and Buffers X 

Purines and Pyiimid X 



Exam Clinical 

Quaflma ^ b1es Emblem 



x 
x 
x 

X 
X 
X 
X 
X 
X 
X 
X 
X 



X 
X 
X 

X 
X 



Note: The exam questions module Is complete for all topics of Medical Biochemistry, including those 
not listed here, such as molecular biology. 



on text in boldfaced type to reveal further 
details, etc. (Baggott, 1990) 

Student attitudes toward CBL and student use of 
IBLS were assessed in two ways. First, survey 
questions were placed on the regular course 
examinations. Several "yes/no" questions 
about previous experience with computers and 
feelings about using 

CBL were included, as well as, for each topic of 
Medical Biochemistry that had an associated 
CBL lecture module (CBL topic), an inquiry as to 
whether the students had learned the topic 
from lecture alone, some combination of 
lecture and CBL, or CBL only. Responses were 
assigned scores of 0, 1 and 2 respectively; the 
scores were used directly to identify type of use, 
and the sum of the scores over all CBL topics 
was taken as a measure of Intensity of CBL use. 
Answers to the survey questions were recorded 
on the examination answer sheets. Answering 
was voluntary, and time for this was provided 
prior to the start of the examination. No more 
than five survey questions appeared on any 
exam nation. As a second source of use 
information, on the first day of the Spring 
semester of 1991, three weeks after the end of 
C-*«? Fall semester, students were asked to 
complete a 22-questlon CBL use survey. This 
survey asked, for each CBL topic, which module 
the student had used: lecture, review and drill, 
examination questions or the clinical 
problem. Possible answers were, "yes," "no," 



and "don't recall." The answer sheet depository 
was placed next door to the lecture hall, and $5 
copy cards were offered to the first 50 students 
returning signed surveys. 

Examination subscores for each student were 
determined for all CBL topics. Comparisons 
were made between performance on CBL topics 
where the CBL lecture module was used vs. 
where it was not, using the paired t-test and 
multiple regression/ correlation. Analyses 
were carried out with Systat version 3.0 

Results 

Student cooperation was nearly 100% while 
the course was in progress; nearly all students 
answered the optional survey questions on the 
examinations. The survey conducted after the 
course was over, however, netted only 93 
responses. Further, these were returned in two 
waves. The first was Immediately after the 
forms were distributed. In fact, some students 
completed the forms while standing in line to 
receive the photocopy card. The second wave 
arrived a day later, after it became known that 
seven cards were still available to be given out. 

Data on computer ownership, prior computer 
experience and attitudes toward CBL were 
collected during a pretest given on the first day 
of the course (Table 2). A minority of students 
had access to a private computer, but most had 
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Table 2 

Personal Factors Related to Use of Computer Based Learning 

The following questions were asked of first year students at the start of the first semester Medical 
Biochemistry course. 'Yes" and "No" were the possible answers. 



Question Percent 'Yes" 

Do you have access to a computer (other than a public 33 
machine)? 

Did you use computers as an undergraduate (for academic 79 
oj recreational purposes)? 

Have you taken any computer courses? 50 

Have you ever used computer assisted instruction? 44 

Do you currently feel favorably Inclined toward using 80 
computer assisted instruction? 



used computers previously, and 80 percent were 
favorably inclined toward CBL. Regression 
analysis revealed that availability of a 
privately owned computer was a significant 
predictor of a favorable inclination toward 
CBL (p ■ 0.008). None of the other variables was 
independently a predictor of attitude toward 
CBL. At the time of the final examination the 
students were again asked whether they were 
favorably Inclined toward CBL. 76 percent 
answered, "Yes," a change that was not 
significant in a paired t-test. 

Data on CBL use collected during the course 
examinations showed In paired t-tests that 
there was no difference between examination 
performance on CBL subjects when the CBL was 
used us. when it was not. Further, multivariate 
analysis with dummy variable coding to denote 
membership In the groups which learned from 
lecture only, from lecture plus computer or 
from computer only showed no effect of group 
membership. That is, in any given CBL subject, 
there were no differences In examination 
performance that could be attributed to 
different ways of learning. 

This surprised us because pilot studies In 
previous years, when there had been far less 
CBL available, had suggested that there was an 
effect of group membership, and that use of CBL 
lecture modules improved examination 
performance. We had expected that with more 
CBL available, this effect would increase, and 
that statistical significance would be achieved. 
Failure to observe this outcome led to suspicion 
that students interpreted the phrase, "learned 
from some combination of lecture and CBL," 
differently this year as compared to previous 
years. In the past, CBL lecture modules were 
separate stand-alone programs, and it seemed 
natural to Interpret the phrase, "learned from 
CBL," as referring to the lecture modules. This 
year, for the first time, all CBL was Integrated 



Into IBLS, and the distinction between primary 
learning from the lecture replacement vs. drill 
or self-testing may have become lost. We had 
known for some time (Dennis, 1988) that 
examination questions were the most popular 
CBL activity with our student population. We 
speculated that this year some students who 
claimed to have learned from both CBL and 
lecture had not used the CBL lecture modules at 
all, and had answered positively on the 
strength of having used only the examination 
questions. This was the reason for the post- 
course survey. 

Comparison of the results of the two surveys 
provided evidence that the hypothesis of 
different Interpretations of the same question 
by students in different years may have been 
correct. Some students who had previously 
claimed to have used CBL to leam a given topic 
indicated subsequently that they had not used 
the lecture module, but had used other modules 
of IBLS for that topic. In addition, some 
students who had previously stated that they 
had NOT used CBL claimed later to have done 
so. We accepted the former at face value, and 
the latter as representing additional CBL 
lecture module use subsequent to the earlier 
survey. Both groups were counted as nonusers 
of the lecture modules at the times of the 
examinations. These adjustments to group 
membership data failed to produce any 
significance in the paired t-tests or in the 
multivariate analyses. 

Discussion of these findings with the manager 
of the Microcomputer Learning Laboratory 
yielded the following observation: He had the 
impression that there was a noticeable 
difference in the way students used the software 
this year in comparison with last year. Last 
year, he felt, they used lecture modules 
Intensely and for prolonged periods, while this 
year use seemed more casual and more 
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abbreviated. This observation, If substantiated 
quantitatively, could represent a difference 
between use of CBL when It Is a novelty and 
routine use when a large fraction of a course Is 
available on the computer. Clearly a way to 
distinguish between casual use and serious 
study was necessary. 

The best reflection available In our data of 
serious study Is the number of CBL topics for 
which a given student used the lecture module, 
modified appropriately for use of CBL Instead 
of attending lecture whenever that occurred, 
This is given by the intensity variable defined 
in the methods section. Interpretation of 
intensity in this manner assumes that students 
who use CBL for many subjects do so because 
they feel it is effective for them, which in turn 
implies that they use it seriously. When this 
variable was used as a predictor of performance 
on CBL topics for which the lecture modules 
were used, it was positive and significant (p = 
0.012). The predictive equation Is 

% correct = 69.8 + 0.7 Intensity 

This means that CBL does have a significant 
effect on exam performance, and that the effect 
is in evidence exactly to the extent that 
students take CBL seriously and use it 
conscientiously. In contrast, Intensity was not 
a significant predictor of performance on 
topics for which no CBL was available (p = 
0.276), nor on topics for which CBL was 
available but not used (p = 0.702). Lack of 
significance where none would be expected is 
consistent with the interpretation that 
intensity is not merely a reflection of overall 
academic diligence, but is specifically related 
to an effect of CBL use. Analogous results have 
recently been reported from a very different 
educational setting (Atkinson, 1991). 

Discussion 

The most reasonable interpretation of these 
results is that buried somewhere within the 
"Intensity" variable is a factor or factors 
bearing a direct causal relationship to 
examination performance. It or they probably 
measure the amount of time and degree of 
seriousness with which the student used CBL, 
and therefore represent a true effect of CBL. 

The effect is both statistically significant and 
practically significant. For each unit of 
intensity, performance on the CBL subjects 
Increases by 0.7 percentage points. Over the 
nine CBL subjects currently available, this 
translates Into a minimum of a (0.7 x 9 ■ ) 6 
percent overall improvement attributable to 
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use of CBL as an adjunct to the lecture. In 
practical terms, this would have meant a 6 
percent upward shift in the grade distribution 
curve, resulting In 20 fewer course failures If 
CBL had been available for all topics and 
everyone had used it. 

That Intensity has predictive value eliminates 
a novelty effect. It Is the users who make a 
sustained effort that achieve the most, 
individuals who are Initially enthusiastic, but 
later use fewer lecture modules (and 
presumably use them less diligently) 
experience a smaller performance Increment 
for the topics in which they used CBL. 

Our results could in principle be accounted for 
by a practice effect. We believe this is not the 
explanation. The components of IBLS all use 
the same conventions and user interface, and 
experience shows that most students become 
comfortable with the mechanics of Its use 
within 15 minutes. What varies from module 
to module Is the subject matter. 

For the most part the lecturer In the course also 
wrote the corresponding CBL lecture modules 
and the examination questions, raising the 
possibility that the examination questions 
could have been biased in favor of the CBL 
users. The lectures and the CBL were designed 
to be identical in content, visuals, etc. 
Examination questions were based on written 
behavioral objectives, which were the same for 
both presentations. Further, the positive effect 
of CBL use suggests that potential subconscious 
cues such as gesture, intonation or repetition 
by the lecturer were not significant. 

We speculate that the Improved performance 
experienced by intense users is not merely an 
effect of spending more total time on the 
material. Students' days are heavily scheduled, 
and they do not have discretionary time. If 
they put more time into CBL it would have to be 
at the expense of some other academic pursuit. 
Given the tendency of medical students to be 
strategic learners (Newble, 1986), we doubt that 
this occurs. This interpretation is consistent 
with the belief that the way in which our CBL 
presents information, with advance 
organizers, animation, self-pacing, etc., 
improves learning. 

Failure of multivariate analysis with dummy 
variable coding to yield the anticipated result 
is probably because under the conditions 
prevailing this year, when a relatively large 
amount of CBL was available, the mere fact of 
use bore no implication regarding seriousness 
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of use. Further, this year very few students 
elected to use CBL Instead of attending lectures, 
although the majority made some use of the 
lecture module for every CBL topic. All of this 
may in turn be related to computer 
availability. Students may have found it 
impossible to use CBL to an optimum extent 
with sx student/computer ratio of 20. Indeed, 
during the first month of the semester IBLS 
accounted for 70% of Macintosh use in the 
Microcomputer Learning Laboratory, and it 
was very difficult to find an available 
Macintosh during hours when the first year 
students were out of the classroom or 
laboratory. 

In principle a randomized study might be 
carried out in which students would be required 
to learn the subject in prescribed ways and be 
required to keep detailed logs of learning 
activities. Such an approach, however, is 
antithetical to our principles of student- 
controlled learning. Also, it would be 
unenforceable upon students spending large 
sums of money to prepare themselves for a 
life's work. 

It would probably be an error, however, to 
make improved performance attributable to 
CBL into a Holy Grail. As long as CBL and the 
lecture are demonstrably equally effective, as 
Haglor and Knowlton (Haglor, 1988) have 
argued they should be if the content and 
organization of the two media are identical, 
CBL still has advantages. It Is always 
available; it is infinitely patient; it will do 
what the user wants when the user wants to do 
it. Further, it can achieve the ideal 
presentation of information, with 
sophisticated graphics and animations that 
are difficult to bring to the classroom, it never 
feels out of sorts (although computers do 
become infected with viruses), and it can be 
linked to related Information in any way the 
program author wishes, thus integrating 
information across disciplines as well as 
within a given discipline. 
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A COMPUTER MANAGEMENT SYSTEM FOR PATIENT SIMULATIONS 



Lynn Johnson, The University of Iowa 



ABSTRACT 

We are teaching clinical problem-solving 
skills, including diagnosis and management, 
to dental students using a series of interactive 
videodisc patient simulations. We call this 
series Oral Disease Simulations for Diagnosis 
and Management (ODSDM). (Flnkelstein, et 
al., 1988) ODSDM presents an oral pathology 
diagnostic schema and thirty patient 
simulations. ODSDM Is a prototype for a 
comprehensive dental simulation project. 
This paper describes a computer management 
system which assigns students patient 
simulations, tracks their progress, and 
generates reports. 

Our computer management system was 
designed In response to the following needs. 
First, the sequence that students perform 
simulations Is critical. Each student should be 
presented with the individualized sequence of 
simulations they require to master clinical 
decision-making skills. Second, maintaining 
records of completed simulations and student 
performance on each simulation is a time- 
consuming task for faculty. Third, the 
simulations and the simulation models 
require ongoing evaluation to ensure high 
quality instruction. 

The objective of the management system is to 
ensure that each student masters diagnosis. 
Mastery must be obtained at a specified level 
before advancing to the next level. The 
management system does this by 
Individualizing the sequence of the 
simulations to adapt to the needs of each 
student. Students who learn diagnosis quickly 
need to complete fewer simulations than those 
students who learn diagnosis less quickly. 

The management system generates 200 
demographic, progress, and system reports. 

INTERACTIVE VIDEODISC PATIENT 
SIMULATIONS 

Dental students graduate with a solid 
knowledge of the basic sciences and the 
technical skills required to successfully treat 
patients. However, time and access to patients 
may limit clinical experiences in applying 
knowledge to the problems they may encounter 
in a normal practice. Students may also lack 



experience in developing a systematic 
approach to gathering information and solving 
diagnostic and management problems. 

Patient simulations have been used 
successfully in teaching problem solving in 
medicine. In the absence of a sufficient 
quantity and variety of patients, simulations 
have enabled medical students to practice 
diagnosis and management, with no risk to the 
patient, and at a reasonable cost. (McGuirre, 
1976; Templeton. 1985; Raj. et al.. 1982; 
Abdulla. et al., 1983; Blancher, 1985) At the 
University of Iowa we have developed a patient 
simulation project titled Oral Disease 
Simulations for Diagnosis and Management 
(ODSDM). (Flnkelstein. 1988) The ODSDM 
package consists of a series of interactive 
videodisc patient simulations that expose the 
student to case studies representing a variety of 
oral diseases. 

ODSDM has been designed to meet the 
following requirements. First, students must 
take an active role in gathering and applying 
information to the formation of a clinical 
diagnosis and appropriate treatment 
recommendations. Second, students must be 
encouraged to develop problem-solving skills 
and to apply them to a broad and increasingly 
complex range of clinical problems. Third, 
these criteria must be met at a reasonable cost. 

ODSDM uses a laser-reflective videodisc 
controlled by microcomputer (Figure 1). The 
student views two television screens and 
communicates via a microcomputer. One 
screen displays videodisc Images while the 
other displays computer text. The system 
allows the student to gather visual and textual 
information on the patient's disease, offer a 
differential diagnosis, and propose treatment. 
The system also provides feedback to the 
student about the problem-solving process. 

The videodisc presents high-resolution visuals 
of the clinical, radiographic, and histologic 
features of a patient's disease. The videodisc 
Images can be randomly accessed almost 
Instantaneously, and are durable and easily 
maintained. 
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Figure 1. An ODSDM Work Station 



PROBLEMS WITH SIMULATIONS 

The patient simulations solved an important 
Instructional problem, but created three new 
problems. First, the sequence In which 
students complete simulations is critical. Each 
student needs to be presented with an 
Individualized sequence of simulations that 
will ensure his or her mastery of clinical 
decision-making skills. The random 
assignment of patient simulations does not 
guarantee this mastery. Second, recording 
completed simulations and student 
performance on each simulation is time- 
consuming for faculty. Third, to maintain a 
high standard of quality, it is necessary to 
continually evaluate and revise simulations. 
Once the patient simulations were integrated 
Into the curriculum, it became clear that the 
record keeping, evaluation, and revision 
processes needed to be streamlined. 

MANAGEMENT SYSTEM 
Our solution to the problems of student 
mastery, record keeping, evaluation, and 
revision consists of a management system and 
database operating on a local area network. 
The management system sequences the patient 
simulations for each student. The database 
holds demographic and performance 
Information on each student. 

The management system organized the patient 
simulations into a series of patient simulation 
models. Each simulation model differs in 
content, difficulty, and instructional 



techniques. Models of differing complexity 
appropriate to differing levels of student 
expertise and experience are defined by an 
Instructor using a word processor. 

A compa ;.. i of the sophomore and Junior 
ODSDM models illustrates the variations in 
level of difficulty. Sophomore students are 
required to diagnose a patient's disease, but are 
not asked to recommend treatment. If the 
student's diagnostic decision Is Incorrect, the 
message "Reevaluate your decision" appears 
immediately, and the incorrect decision 
becomes unavailable. The Junior model 
requires students to diagnose and propose 
treatment for a patient's disease. The 
"reevaluate" message Is turned off. The student 
is informed of all correct decisions later In the 
Critique. 

The primary goal of the management system is 
to ensure that each student master diagnosis. 
To accomplish this goal, the management 
system is told Which students are assigned to 
each model. During a student session, the 
management system presents a patient 
simulation. If the student successfully 
completes a simulation, one of the model's 
requirements is "checked off' and another 
simulation is presented. When the student has 
"checked off' all of the model's requirements, 
he or she has met the requirements for 
mastery. 'When the students "fall" two patient 
simulations, they are remediated with an 
easier simulation, or an equivalent simulation 
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with Instructional prompts. Once the student 
successfully completes the remediation work, 
he or she is again presented with simulations 
that will fulfill the model's mastery 
requirements. This "check-off" and 
remediation process permits the system to 
adapt to the needs of the individual student. 
Those students who learn diagnosis quickly 
need to complete fewer simulations than those 
who learn less quickly. 

SEQUENCE OF SIMULATIONS 

The management system also individualizes 
the sequence of the simulations. The author of 
a simulation model specifies whether the 
patient simulations are presented randomly or 
in a specific sequence. We have specified our 
sophomore model to present simulations in a 
specific sequence because novices learn more 
efficiently with a structured approach. We have 
specified our Junior model to present 
simulations randomly to challenge the 
developing problem-solving skills of the more 
mature student. 



RECORD-KEEPING 

The database associated with the management 
system stores demographic and performance 
information on each student and uses this 
information to generate reports about 
individual students, or groups of students. The 
reports allow us to track student progress. 
Those students requiring remediation can be 
identified for Individualized attention. 

EVALUATION 

At the end of each simulation, students may 
comment on the simulation's strengths and 
weaknesses. At regular intervals, students 
complete forms evaluating a patient 
simulation. These comments and evaluations 
are stored in the management system and may 
be printed on request. These evaluation reports 
act as a quality control mechanism for the 
patient simulations and the simulation 
models. The student reports and the evaluation 
reports may be used to generate research data. 



Local Area Network (LAN) 




Printer 



Figure 2. A Local Area Network (LAN) Containing Four ODSDM Work Stations. 
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HARDWARE 

A local area network (LAN) allows a group of 
computers to send messages to one another and 
use the same programs. The LAN links the 
student work stations to a central 
microcomputer called a file server (Figure 2). 
The file server contains all the software for the 
patient simulations, the management system, 
and the database with its associated 
information. The LAN also enables the work 
stations to share a single printer. The LAN we 
use is Ethernet, made by 3 COM Corporation. 
Ethernet has a single coaxial cable linking the 
file server, work stations, and printer. 

The computers linked by a LAN must be similar 
for the system to operate efficiently. We use an 
IBM PS/2 Model 70 for the file server, and IBM 
PS/2 Model 30's, each with a Pioneer LD-V6000 
videodisc player, for the student work stations. 



SOFTWARE 

Our system utilizes multiple software packages. 
The user does not see or handle the software 
when performing simulations. 

The most basic software is DOS, which is used 
to add or delete models and patient 
simulations, and to edit patient simulations. A 
word processor is used to author and modify 
the simulation models and their associated 
patient simulations. 

The LAN runs on a software package called 
Novell. Novell controls access to ODSDM and 
the videodisc. Students have limited access 
rights. They cannot 

modify simulations, or demographic and 
performance information. Novell also 
controls sharing the simulation models and 
patient simulation files, and coordinates 
printing. 

The management system uses a database called 
Paradox. Paradox allows us to store and 
manage student demographic and performance 
information, and all simulation comments 
and evaluations. It also is used to generate 
reports. 

The patient simulations use three software 
components: the simulation driver program, 
three baseline files, and individual patient 
files. The driver program presents 
information from the baseline and patient 
files, permits movement within the program, 
scores the student's work, and controls the 
sequence of computer and videodisc material. 
The baseline files contain the information that 



is common to all patients, while each patient 
file contains the information specific to the 
individual patient. 

The management system software coordinates 
a student's movement between DOS, Novell, 
Paradox, and the simulations. It also assigns 
patient simulations to students based on the 
simulation model requirements and 
Individual student progress. 

MAINTENANCE OF THE MANAGEMENT 
8YSTEM 

Maintenance of the management system 
requires a significant amount of effort and 
skill. Someone familiar with the system 
hardware and software must be available as a 
trouble-shooter when problems arise. Our 
experience has revealed that clear, detailed 
software documentation is a necessity. As a 
result, we are presently writing the 
management system documentation. On-going 
tasks Include entering student demographic 
information; performing back-ups of patient 
simulations, simulation models and stored 
data; and generating reports. The management 
system permits clerks to perform these tasks 
rather than faculty. 

COST OF THE MANAGEMENT SYSTEM 

The cost of the management system includes 
designing, programming, and pilot-testing the 
software. The total cost of development was 
$58,700. This Includes programming: $28,000; 
faculty and instructional designer time: 
$22,700; hardware: $5600; software: $1400; and 
documentation: $800. The cost effectiveness of 
the management system derives from the 
reduction in faculty time required for record- 
keeping and simulation evaluation, as 
mentioned above. 
FUTURE DEVELOPMENTS 
An extensive summative evaluation of the 
ODSDM project is planned for the coming 
school year. The management system will be 
evaluated as part of that project. 

The management system has been integrated 
into the oral pathology curriculum, and will 
eventually be a part of a broader dental 
simulation project called Dental Diagnosis and 
Treatment (DDx&Tx), which will encompass 
clinical decision making skills for diagnosis 
and treatment planning in all the dental 
specialties. 

We have also developed Dental Diagnosis and 
Treatment: A Videodisc Atlas. This videodisc 
contains approximately 25,000 slides from ten 
dental and oral health care disciplines, a 



radiograph and histology lmagebase, and 
motion and audio Information representing 
patients' diseases. We are indexing audio and 
visual information Into a database. Plans are 
underway to produce a second edition of the 
videodisc. We are also developing a videodisc 
testing project and a presentation program for 
lectures and Individualized instruction. 
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DEVELOPMENT OP COMPUTER EDUCATION IN HEALTH SCIENCE 
(NURSING); SOME AUSTRALIAN EXPERIENCES 



Nitya L Karmakar, University of Western Sydney 



ABSTRACT: 

University-based nursing education is new in 
Australia in New South Wales, diplomas in 
nursing started in 1985 and universities are 
now offering Bachelor degrees. Computing has 
not been significant in these rapidly developed 
curricula. This paper looks at Important issues 
in introducing computers in nursing education, 
because the nursing profession in Australia is 
challenged by changes in information 
technology. 

INTRODUCTION: 

Information Technology is reshaping our 
society day to day in all aspects of our lives. It 
has become an integral part of patient care in 
all clinical settings. As a result of the 
emergence of new technology the Australian 
Nursing Curricula will require restructuring so 
that our nurses can meet future technological 
challenges. We should put greater emphasis on 
nursing applications of computing in the 
curricula. 

The need to introduce computing into nursing 
curricula has been discussed by several authors 
(Robinson. 1982. Happ. 1983, Hasset, 1984. 
Chang. 1984. Andresli and Musser 1985. 
Walker. 1986. Soya and Lentz. 1987. Staggers. 
1988, Delaney, 1989. Summers etal. 1990 and 
Dover and Boblin, 1991). 

The nursing profession was revolutionised by 
Florence Nightingale, who was also a pioneer 
in the Introduction of scientific methods in 
nursing. "Nursing's evolution as a science can 
be traced back to the days of Nightingale" 
(Mallnski. 1986, p.35). 

The idea of introducing computing as a 
compulsory subject has generated considerable 
interest and concern among some Australian 
Nursing educators and students (Karmakar 
1991). 

Grobe (1984) also expressed some concern for 
the future. 

"Our graduates cannot be thrown into the 
technological maze without under-standing the 
nature of Computer Systems as powerful 
technological tools, an experience base that 



Includes competency In these tools, and a clear 
recognition of the ethical, professional and 
legal implications that accompany computer 
system adoption and use" (p.97). 

Computing in Nursing, does it make a 
difference? My answer to this critical question 
is yes. Change occurs even in the most 
conservative societies. Health care is no 
exception to this. Information technology has 
become an integral part of patient care in all 
modem health care settings. We are now a part 
of a high technology society based on 
information exchange. 

LANDMARK EVENTS OF COMPUTERS INTO 
NURSING 

Computerised nursing education is very 
popular in the US, but this is not so in 
Australia. Several landmark events have 
occurred over the past 17 years. All have 
promoted computer awareness among nurses 
(Table 1). 

NURSING COURSES AT AUSTRALIAN 
UNIVERSITIES 

The transfer of undergraduate nursing 
education from hospital to tertiary 
institutions in Australia is relatively new. 

On November 7, 1983 the then New South Wales 
Minister for Health, Mr Laurie Brereton, 
announced the transfer of basic nurse 
education from hospitals to Colleges of 
Advanced Education from January 1, 1985. As 
Mr Brereton stated, "I believe this change to be 
the greatest advance in nurse education in the 
history of nursing in this country". In the 
following year it was announced that such a 
transfer would be complete throughout 
Australia by 1993. 

NSW Is pioneering an University based nursing 
education in Australia. In all other states, the 
responsibility for basic nursing education 
from hospital to course is not yet complete. In 
NSW colleges of Advanced Education no longer 
exist. All Australian tertiary institutions will 
soon either be an independent University or a 
part of an established University. 



Table 1. 



Landmark Events of Computer In Nursing 



Year 



Title 



Place 
"Fairfax? VA~' 



1973 Invitational conference on 

management information systems 
for public and community health 
agencies 



1974-75 



1976 



1977 
1979 
1980 

1981 
1981 

1982 
1982 



1982 
1982 

1982 
1983 
1983 

1983 



Five workshops on management 
informaUon systems for public 
and community health agencies 

State of the Art conference 
in management for public and 
community health nursing 
agencies 

Research conference on nursing 
information systems 

Trimis conference on computers 
in nursing: A user's perspective 

Early workshop on computer usage 
in health care: A national 
survey 

First National conference on 
computer technology and nursing 

Fifth Annual symposium on 
computer applications in 
medical care (SCAMC-5) 

Study group on nursing 
Information systems 

International meeting: 
Working conference on the 
impact of computers on nursing 



Second National conference on 
computer technology and nursing 
Sixth Annual symposium on 
computer applications to 
medical care (SCAMC-6) 

First newsletter - Computers in 
Nursing 

Medinfo-83: Fourth World 
Congress on medical informatics 

Third National Conference 
on computer technology and 
nursing 



All over USA 



Washington, DC 



Chicago 

Washington, DC 
Akron, OH 

Bethesda, MD 
Washington, DC 

Cleveland, OH 

London and 
Harrogate, 
Yorkshire, 
ENGLAND 

Bethesda, MD 

Washington, DC 

Austin, IX 
Amsterdam 
Bethesda, MD 



Second annual Joint congress San Francisco 
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1983 
1984 

1984 

1984 
1984 
1984 
1984 
1985 



1985 
1985 
1986 
1986 

1987 

1987 

1987 
1988 



and conference 

Newsletter - Computer in Nursing 

Fourth Annual conference on 
computer technology and nursing 

Eighth annual symposium on 
computer applications In 
medical care (SCAMC-8) 

Third Joint National Congress 
and conference 

First Journal - Computers In 
Nursing 

Council on computer application 
In nursing (CCAN) 

National forum on computers in 
health care and nursing 

The International symposium 
on nursing use of computers and 
information science, "building 
bridges to the future". 

The first annual nursing 
interactive video conference 

On-Line - Off-Line: computer 
use in nursing 

The 5th World Congress on 
medical informatics (MedUnfo 86) 

The Fifth National conference 
on computer technology and 
nursing 

The 1st World symposium on 
computers in care of the mother, 
foetus and newborn 

Computer applications In 
medicine and health care 
(AAMSI Congress 1987) 

The Nursing Informatics 
Symposium 

Nurses and computers, an 
international symposium on 
nursing, computer use and 
Information science 



& Baltimore 
Philadelphia 
Bethesda, MD 

Washington, DC 



San Francisco 
& Washington, DC 

Philadelphia 



Kansas City, MD 
New York 
Calgary, Canada 



Sacramento, 
California 

Atlanta, 
Georgia 

Washington, DC 
Bethesda, MD 

Vienna, Austria 



San Francisco, 
California 



Columbus, Ohio 



Trinity College, 
Dublin 
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1988 



1989 



1989 



1990 



1991 



The Fourth annual computers In 
nursing symposium: applications 
in practice and management 

Medical informatics and 
education: an international 
symposium 



University of 
Massachusetts 
Medical Centre 

British Columbia, 
Canada 



The Thirteenth Annual Symposium Washington, DC 
on computer applications in 
medical care 

The Fourteenth Annual Symposium Washington, DC 
on computer applications in 
medical care 



Computers and nursing practice, 
6th Annual seminar focusing on 
mainframe and micro computer 
applications (May 23 & 24) 

The Fourth International 
conference on nursing use of 
computer and Information 
science (April 14-17) 



NYV Medical 
Centre. USA 



Melbourne, 
Australia 
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Table 2 



SUMMARY OF AMALGAMATIONS OF TERTIARY INSTITUTIONS 
IN NSW AND ACT 1990/91 



Australian Catholic 
University (NSW & ACT 
Campuses) 



Incorporating: Catholic 
College of Education, Sydney 
Slgnadou College 



Australian National 
University 



Will Incorporate Canberra 
institute of the Arts In 1992 



Charles Sturt University 

Rlverina-Murray Institute 
of Higher Education (IHE) 

Macquarle University 
Institute of Early 
Childhood Studies 

University of Canberra 

University of Newcastle 



University of New South 
Wales 



University of New England 



University of Sydney 



University of Technology, 
Sydney 



University of Western 
Sydney 



University of Wollongong 



Incorporating: Mitchell College 
of Advanced Education (CAE) 



Incorporating: Sydney CAE 



Formerly Canberra CAE 

Incorporating: Hunter IHE 

NSW State Conservatorium of Music, 

Newcastle 

Incorporating: College of Fine 

Arts, (formerly City Art Institute). Sydney CAE 

- St George Institute of Education 

Incorporating: Armldale CAE, Northern Rivers CAE, 
Orange Agricultural College 

Incorporating: Cumberland College of Health Sciences. NSW 
Institute of the Arts - Sydney College of the Arts. NSW State 
Conservatorium of Music, Sydney Sydney CAE - Institute of 
Nursing Studies. Sydney CAE - Sydney Institute of Education. 
Sydney College of the Arts. 

Incorporating: Kuring-gal CAE. 

Sydney CAE - Institute of Technical and Adult 

Teacher Education 

Incorporating: Hawkesbury 
Agricultural College. Macarthur IHE, 
Nepean CAE. 
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Table 3 



SUMMARY OF NURSING COURSES {UNDERGRADUATE) AVAILABLE 
IN NSW/ACT UNIVERSITIES 



University 



Offered 
by 



Mln. 
duration 
(years) 



Pattern 
of attend. 



Name of the 
course 



Australian 

Catholic 

University 



Division 
Nurse 
Education 



of 



3/6 



Full tlme(F)/ 
Part time(P) 



Diploma of 
Health 
Science 
(Nursing) 



Charles 
Sturt 

University 



School of 

Nursing 

& Health 

Admlnlst. 

Mitchell/ 

Rlverina 



Diploma of 
Health 
Science 
(Nursing) 



University 
of 

Canberra 



Department 
of Nursing 
School of 
Applied 
Science 



3/6 



F/P 



Diploma of 
Applied 
Science 
(Nursing) 



University 
of New 
England 



School of 

Nursing 

Armldale 



Northern 
Rivers 



3 

3/6 



F 

F/P 



Diploma 
Health 
Science 
(Nursing) 

Dip.Health 
Science in 
Nursing 



University Faculty of 3 F Diploma 

of Health of Health 

Newcastle 2? icn ? c , 

(Nursing) 



University 
of Sydney 


Faculty of 
Nursing 


3/6 


F/P 


Diploma of 
Health 
Science 
(Nursing) 




Cumberland 
College of 
Health 
Sciences 


3 


F 


Diploma 
of Applied 
Science 
(Nursing) 
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University Faculty of 3 F Diploma of 

of Nursing Nursing 

Technology 
Sydney 



University 
of Western 
Sydney, 



Hawkesbury 
Faculty of 
Nursing and 
Community 
Studies 

Macarthur, 
The School 
of Nursing 
and Health 
Studies 



3 



Diploma of 
Health 
Science 
(Nursing) 



Diploma of 
health 
Science 
(Nursing) 



Nepean, 
Faculty of 
Health 
Studies 



3/6 



F/P 



Diploma of 
Health 
Science - 
Nursing 



University Faculty of 3 F Diploma of 

of Health and Nursing 

Wollongong Behavioural 
Science 



Our discussion concentrates on Universities in 
NSW and Australian Capital Territory (ACT). 
The University of Sydney, founded In 1850, was 
the first university to be established in 
Australia. A number of tertiary Institutions 
were amalgamated In 1989 

(Table 2). There are about 27,000 academics 
employed by all tertiary Institutions in 
Australia. 

The Nursing diploma courses available at 
Universities are shown In Table 3 (course 
handbooks, 1991). 

The Diploma Course comprises 3 years of full- 
time study or 6 years of part-time study. The 
subjects consist of physical and biological 
sciences and the basic principles and processes 
of nursing. 

Theoretical studies are usually conducted at the 
University campuses whilst clinical 
experience, which accounts for approximately 
50% of the course Is provided in community 
health centres, Early Childhood Centres, 
Nursing Homes, Schools and Hospitals for the 
Developmentally Disabled, General Hospitals 
for children and adults and Psychiatric 
Hospitals. 



At the completion of the three year diploma 
students and registered nurses from hospital 
based certificate level courses may wish to 
continue their study full-time or part-time to 
gain a Bachelor of Nursing degree, which Is 
limited to a few universities. The duration of 
this course varies from 6 months to 2 years 
full-time or 12 months to 4 years part-time. 

There are a little over three thousand students 
enrolled In Diploma Courses at NSW and ACT 
Universities, but only about 150 students are 
enrolled In Bachelor Courses. The Bachelor of 
Nursing Courses are gaining popularity among 
students. This is encouraging other 
participating universities to offer bachelor 
degrees In nursing. 

Successful completion of the Diploma enable 
the graduate to register with the relevant State 
Health Registration Board and practice In a 
variety of clinical areas, including general 
medical and surgical, psychiatric, 
developmentally disabled, paediatric, geriatric 
and community health. 

COMPUTERS IN NURSING CURRICULUM 

Computer technology is playing an 
Insignificant role in nursing education in 
Australia. But some new universities have 
eagerly pursued the introduction of the 
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computer technology In the curriculum. The 
University of Western Sydney. Hawkesbury is a 
pioneer in introducing Computing as 
compulsory subject. This is not the same in 
other more conservative and traditional 
universities. In fact, the author could not 
locate another university In Australia where 
computing Is taught as a compulsory subject. 

The author is Involved In teaching computing 
to Nursing students. The aim. objectives and 
course contents are as stated below. 

UNIVERSITY OF WESTERN SYDNEY. 
HAWKESBURY 

FACULTY OF BUSINESS AND LAND 
ECONOMY 

NUR 433 - COMPUTER APPLICATIONS IN 



COURSE OUTLINE - AUT UMN SEMESTER 
J221 

LECTURER: Dr Nltya KARMAKAR 

AIM: 

To develop nurses with knowledge, skills, 
understanding and applications of computer 
technology. 

OBJECTIVES: 

Upon completing this component the student 
will be able to: 

•understand the basic attributes of the three 
most commonly used application software 
packages; 

•describe the advantages of word processing 
over typing, spreadsheets over ledger sheets 
and electronic databases over filing systems: 

•define integrated software and describe the 
advantages of integrated packages over single 
task packages: 

•explain what to look out for when purchasing 
packaged software: 

•develop confidence in their ability to 
successfully interact with micro-computers 
and use them in a variety of situations: 

•provide opportunities to see computer 
technology as an important tool in the 
solution of problems in nursing: 
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•demonstrate the practical skills In the use of 
the three popular applications software 
packages. 

TOPICS: 

A. Fundamentals of Computing 

• The Evolution of Computing 

• Hara»vare 

• Input and Output 

• Software 

• Operating systems 

• Software development 

• Social Issues and Technological Trends 

• Computer Applications In Nursing 



B. Application Packages 

• Word processing 

• Spreadsheet 

• Database 

t 

DURATION: 

2 hours per week for 14 weeks (one semester). 

Most of the students are very positive about 
studying computing. 

RECOMMENDATIONS: 

The Introduction of computing into nursing is 
essential In relation to all of the following 
areas: 

• Nursing practice 

• Nursing standards 

• Nurse supply 

• Nursing services 

• Nursing education 

• Nursing research 

"The Nursing Curriculum needs to be reviewed 
very urgently to prepare our nursing students 
for the 21st century" (Karmakar, 1991). The 
critical question is: 'Has computing, as it has 
advanced during the last decade, influenced 
and brought about changes in nursing in 
Australia?' The answer is no. Computing has a 
very insignificant role at present in the 
nursing profession and education in Australia. 




There Is some change In the curriculum at the 
more progressive Universities, but the pace of 
change at other Universities is frustratingly 
slow. 

A strong and diverse nursing education and 
service will be based on the following 
developments: 

•Designing a suitable computing curriculum for 
all nursing students at Australian 
Universities. 

•Development of the curriculum being based on 
interviews with Australian nurse educators, 
questioning of graduate nurse students and 
review of Australian health needs, nursing 
education and practice, nursing 
administration and research. 

•Search for successful models to follow 
especially reviewing the development of 
University nursing education In the US and 
European countries. 

•Available funds for in-service training for all 
nurse educators who have not had any 
training in computing. 

•Development of specific training curricula for 
all practising nurses to become computer 
literate. 

We can learn what not to do from the United 
States and to some extent from Britain. 

CONCLUSION: 

The transfer of nursing training from the 
hospital system into Universities to Australia 
has warranted the need to develop curricula 
with the introduction of computer into nurses' 
learning and practice environments to prepare 
them for careers in our increasingly 
computerised health care settings. Most of the 
existing courses have strong emphasis on 
physical, social and biological sciences. 
Recently Australian Nursing students have 
complained about the structure of their 
curricula being too academic (Throp. 1990). 
The nursing curricula should reflect the 
combination of the best of traditional values in 
teaching with the latest innovations in thought 
and technology. 
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CAD INSTRUCTION IN INTERIOR DESIGN PROGRAMS 



Stephanie demons, M.S„ Colorado State University 
Joan McLain-Kark, Ph,D„ Virginia Polytechnic Institute 



Abstract 

In May 1990, the Interior Design Educator 
Council (IDEC) membership was surveyed to 
determine the portion of interior design 
programs that have incorporated CAD into 
their curricula and other pertinent 
Information. The responses Indicate that of 
the IDEC Interior design programs nationwide, 
an Impressive 89 percent have Incorporated 
CAD Instruction. 

Introduction 

Although the national design organization 
memberships of practicing interior designers 
have been surveyed to assess their attitudes 
toward CAD training for Interior design 
students and to assess their usage of CAD, little 
data has been complied to determine the extent 
to which CAD Instruction is being offered 
nationally through interior design programs. 
Institutions across the nation thst teach 
Interior design are adding CAD classes to their 
curricula with the assumption that the courses 
will enhance graduates' marketability. 

As the Implementation of CAD as a design tool 
continues, many new issues and challenges are 
becoming evident for the Interior design 
education field. Some of the questions this 
research and the following discussion begin to 
explore are: When should CAD Instruction be 
offered? How does the timing of CAD 
instruction relate to the sophomore review 
board? How does an instructor remain current 
on the software he/she is teaching? 

Purpose of Study 

In May, 1990, the Interior Designers' Educators 
Council (IDEC) membership was surveyed to 
determine the proportion of interior design 
programs that have incorporated CAD and to 
obtain other pertinent Information concerning 
CAD Instruction. By studying both practicing 
Interior designers' attitudes and usage of CAD 
as well as educators' implementation of the 
design tool, one can obtain a clearer 
understanding of the stage in the adoption 
process CAD Is currently at in industry and 
education. One can also estimate the level of 
CAD education design students are receiving 
and If the CAD needs of the design industry are 
being met or surpassed by interior design 



graduates. 
Background 

A 1985 study of practici g Interior designers' 
attitudes, American Society of Interior 
Designers (ASID) membership, revealed that 
only seven percent of the responding designers 
used CAD (McLain-Kark & Tang, 1986). In 1987 
a random sample of the ASID membership 
revealed that 60 percent of the responding 
designers believed that CAD was needed as a 
tool in the design field. Eighty-five percent 
agreed that design students should be trained 
on CAD during their college or design school 
education (Clemons & McCullough, 1989; A 
1989 poll of the Southwest Region membership 
of IDEC found that, of the respondents. 72 
percent of the interior design programs require 
at least one CAD course (Gartska, 1990). 
Clearly, usage of CAD as a design tool is 
growing rapidly. 

Methodology 

A survey of the IDEC membership was 
conducted in May. 1990 In order \o Investigate 
how many interior design programs require 
students to have some CAD instruction before 
graduation. Surveys were sent to 216 U.S. 
design institutions listed in the IDEC 
Membership Directory 1898-1990. The surveys 
-vere addressed to the department heads of each 
in'.ri r design program. Usable responses 
f. ere received from 153 (71%) of the schools. 

The one page survey requested Information 
about the respondents' positions, and whether 
their programs ofTered CAD instruction. For 
those offering CAD, additional information 
requested Included who taught the course(s), 
when It was offered (level of student), proximity 
of the facilities and the amount of time spent 
giving CAD instruction. Responses were coded 
for computerized statistical analysis. 
Frequency distributions and descriptive 
statistics were derived using a statistical 
computer program. 

Seventy-one percent of the surveys returned 
were completed by the department head or 
program coordinator at that school. The 
remainder were filled out by instructors who 
taught CAD courses (7%). another interior 
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design faculty member (14%), or by others (8%). 
Results 

Of the IDEC interior design programs 
nationwide, an Impressive 89 percent (136 
schools) have Incorporated CAD Instruction. 
iSee Figure 1). Of those offering CAD, 63 percent 
Indicated CAD instruction was offered as part 
of a required course. (See Figure 2). A few, four 
percent, voluntarily responded they offer both 
required and optional CAD courses. As this 
option was not available on the survey, the 
actual percentage for schools offering both 
required, as well as optional course, may be 
higher. 

Eighty-two interior design programs (over 60 
percent) had an interior design faculty member 
teaching CAD. (See Figure 3). About 12 percent 
of the CAD Instruction was taught by architects 
and, interestingly, 11 percent by industrial 
science instructors. Responses In an 'other' 
category included engineering (3%) and 
technology education (2%), while apparel 
design, agricultural science and architectural 
drafting each had one percent of the programs' 
instruction. 

CAD Instruction begins at varying levels in the 
surveyed programs. (See Figure 4). Only 10 
percent of the programs begin CAD instruction 
during the freshmen year, it is more common 
at the sophomore and Junior levels. CAD 
Instruction begins at the sophomore year in 37 
percent (50 schools) of the programs, with 
almost an equal percentage, 36 percent (49 
schools), beginning their instruction during 
the Junior level. Only 16 percent of the 
programs begin teaching CAD during the senior 
year. Several respondents explained that CAD 
was Introduced, in their programs, in i all 
segments during the freshmen year and then 
included in various studio projects throughout 
the courses offered before graduation. 

Time spent giving CAD instruction also varied. 
(See Figure 5). Fifty-one percent or 110 
programs offered a one semester CAD course. 
Twenty-two percent of the programs required 
more than a semester or quarter, while 16 
percent of the programs spent a quarter 
teaching CAD to their interior design students. 

CAD courses were taught within 57 percent of 
the interior design facilities while 32 percent 
were taught in facilities across campus. (See 
Figure 6). Several respondents (7 percent) 
indicated that the facilities were across campus 
but that they were considered interior design 
facilities. 



Discussion 

CAD Issues for Design Education 

Sophomore Review Board 

Recently, it has been observed that certain 
Interior design students who are not 
exceptional at producing ideas with traditional 
design tools (T-square, pencil, pen and 
markers), excel using a CAD system as a 
communication device. In the past their 
ability to express their Ideas and design 
solutions may have been hampered by the tools 
available and their insecurity with the skills to 
use them. 

Proper graphic presentation skills count a 
great deal In education, as In the design 
profession. Nationally, many interior design 
programs select a portion of top students to 
continue In their upper division program. 
Usually, a "sophomore review board" evaluates 
students' abilities by reviewing complied 
portfolios and taking grades Into 
consideration. Often, the portfolio Is 
comprised of projects displaying the student's 
design and drawing skills using "traditional 
tools" since CAD Instruction is typically 
Introduced during the Junior/senior levels. 
The student is expected to demonstrate abilities 
to communicate design ideas using traditional 
tools. If the Board decides a student lacks these 
skills, he or she are Is not allowed to continue 
in the program, and subsequently, in the design 
field (Whiteside-Dickson & Rothgeb, 1989). If 
we assume that some students' abilities to 
express design solutions have been hampered 
by the available tools, interior design 
programs may need to rethink their decision 
criteria, portfolio content, and/or placement of 
CAD instruction. 

Individual student attitudes and personality 
traits, may also be part of the reason the 
computer is easier and, therefore, more 
desirable to use than the typical design tools. 
With the increased use of CAD, we may discover 
that students who were not "artsy" enough to 
competently communicate their ideas using 
typical design tools, excel with computer 
graph! « Their ideas and designs, goals to be 
designers, and needs can be expressed through, 
what Is still labelled a non-traditional 
medium. Individuals with personality traits 
more compatible with computer usage may 
surprise us with the quality designs they 
produce. One might say that a communication 
gap has been breached. An entire new breed of 
designers may be on the horizon. 



CAD Instruction in ID Programs 




Figure 1 



Required/Optional Courses 




Figure 2 



Instructors of CAD 




Figure 3 



Level CAD Instruction Given 




36% 



Figure 4 
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Time Spent on CAD Instruction 



SEMESTER 
51% 





MORE THAN A SEMESTER 
22% 





1-4 WEEKS 

5% 

4-8 WEEKS 
6% 



QUARTER 
16% 



Figure 5 



Location of CAD Course 




Figure 6 
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Continuing Education for Design Educators 

Continued education of faculty is another Issue 
to be addressed with the Implementation of 
CAD Instruction. New releases of software 
packages appear on the market annually, If not 
more frequently. Thus, incentives such as 
release time or funds for continuing education 
may need to be given by the administration so 
that CAD Is implemented in other courses and 
programs. A successful example of this is 
Purdue University's Interior Design Program 
Unman, 1990). 

The department head allowed each semester, 
two faculty members release time to develop a 
two-week module of CAD for Implementation 
into an existing course. CAD Instruction was 
successfully implemented from the freshmen 
to senior levels In studio and lecture classes. 

A college, university or area "user group" might 
be one solution for keeping abreast of new 
software releases. Continuing education 
courses at nearby community colleges during 
the summer may benefit both faculty and 
community college personnel. Annual 
conferences and workshops, such as ADCIS's, 
are other avenues for remaining current on 
software /hardware revisions, assignments, 
and new teaching techniques. 

Summary 

This study investigated the prevalence of 
interior design programs nationwide that 
offered CAD instruction to their students. It is 
apparent that the adoption process of CAD as a 
design tool, despite its controversial 
disadvantages, is escalating in design 
education in an attempt to meet or surpass 
needs of the field. 

CAD has become an important part of interior 
design curriculum as indicated by the high 
level of interest in the study expressed by the 
number of surveys returned (71 percent) and the 
percentage of schools incorporating CAD (39 
percent). The majority of instruction is being 
offered by Interior design faculty for a semester 
during the sophomore or junior level. Ttiis 
Indicates that interior design educators have 
shown leadership developing a crucial 
component of their curriculum. 

With the use of CAD in interior design 
education, many issues are developing which 
need our attention. We may discover that 
design students who were not competent to 
communicate their ideas using typical design 
tools, excel with computer graphics. The make- 
up of the design community may change as 



those typically unskilled with traditional tools 
grasp the Intricacies of computer graphics and 
enter the design profession. 

Technology, when used appropriately, can 
improve teacher efficiency, creativity, 
productivity, and professionalism. Emphasis 
must be placed on the ongoing education of 
faculty to fully take advantage of the tools 
currently available and being planned for the 
future. (Charp, 1989). The interior design field 
is counting on the graduates produced from 
these programs to act as catalysts in the 
implementation or continuation of CAD usage 
within their firms. 

References 

Charp, S. (1989, June). Editorial. Technical 
Horizons In E ducation Journal. 16(10), p. 8. 

Clemons, S. A. and McCullough, J. (Fall, 1989). 
Interior designers' attitudes toward CADD and 
CADD education. Journal of Interior Design 
Education and Research . 15(2), pp. 29-34. 

Garstka, J. (1990). A Preliminary Assessment 
of CAD Use in Interior Design Education and 
the Profession. Proceeding s of the Interior 
Design Educators Council Conference , pp. 54- 
55. 

Inman, M, Davis, D., Kilmer, W. O., Kilmer, R, 
Peaslee, J., Postero, T., Willis, V., & Klngsley, 
W. (1990). Computer-aided design in every 
studio classroom. Proceedings of the Interior 
Design Educators Council Conference , pp. 69- 
71. 

McLain-Kark J. &Tang, R E. (1986). Computer 
us<»ge and attitudes toward computers in the 
interior design field. Journal of Interior 
Design Educa tion and Research. 12(2), 25-32. 

Rogers, E. M. & Shoemaker, F. F. 
Communication of Innovations . New York: 
The Free Press, 1971. 

Whiteside-Dickson, A. & Rothgeb, T. (1989). 
Portfolio reviews as a means of quality control 
In Interior design programs. Journal of 

Interior Design Education and 

Research. 15(2). 21-28. 



ERIC 



250 ?f><; 



INTEGRATING COMPUTER AIDED DESIGN AND DRAFTING (CADD) 
INTO INTERIOR, APPAREL AND DISPLAY DESIGN COURSES 



Diane M.Davis, Valparaiso University 



ABSTRACT 

CADD applications In interior design, fashion 
design and visual merchandising are a growing 
reality. Students entering these professions 
therefore must be CADD literate. To help 
students meet this challenge, the author has 
chosen an approach of integrating computer 
experiences into existing coursework with the 
objective of using the computer as a design tool 
rather than as a specific computer system to be 
mastered. 

INTRODUCTION 

CADD applications in interior design, fashion 
design and visual merchandising are a growing 
reality. Consequently, educators are faced with 
multiple demands. One, they must learn to use 
the computer and its myriad of software 
applications; two, they must help students 
develop necessary computer skills to prepare 
for the Job market; and three, they can offer 
retraining for professionals who were not 
exposed to CADD in their formal educational 
training. 

Faced with these challenges, it seemed 
inappropriate to train students or myself on 
only one specific hardware and software 
system since the application needs vary 
between and within each discipline. Also, time 
does not stand still for the computer industry 
and the one thing we can be sure of is short- 
term obsolescence for whatever systems we 
learn and teach. 

I therefore made the decision that I wanted my 
students to learn how to learn to use computers 
so that they would have the confidence to learn 
whatever system the work world presented. To 
meet these objectives, CADD projects were 
integrated into a major portion of their design 
courses. In all cases, one of the main objectives 
of each project was to approach the computer as 
a design tool. Therefore, project concepts were 
determined first and the commands necessary 
to execute the projects were identified second. 
This approach ensured that the emphasis was 
on using the computer as a tool in the design 
process rather than learning specific hardware 
and software first and applying that knowledge 
later. The majority of projects were completed 



within 20 instructional hours. A moderate 
amount of additional hours out of class were 
necessary in most instances. 

A variety of computer software and hardware 
was used for the projects, and these decisions 
were usually determined by availability. Most 
of the projects, however, could be done on 
several different hardware -software 
combinations. Projects relying on drafting 
features were done on either Macintosh with 
MicroStation by Intergraph or on a PC with 
AutoCad. Projects relying on graphics features 
were done on Macintosh with either FreeHand 
by Aldus or Illustrator by Adobe. 



INTERIOR DESIGN 
Background 

The use of CADD in architectural design has 
long been documented in trade and academic 
Journals. Now its application in the interior 
design profession is also being documented. 
Loebelson (1988) reported the percentage of top 
interior design firms using CADD continued to 
increase dramatically from 49% in 1986, to 
58% in 1987, and 69% in 1988. In 1989 the 
trend continued upward to 92% for the the top 
100 firms. (Lobelson, 1989) 

Educators are looking to the profession to help 
develop necessary programs for training 
students. In a study to assess the need for 
CADD training in interior design programs, 
Clemons and McCullough (1989) reported that 
90% of a random sample of members of the 
American Society of Interior Designers felt 
that interior design students upon graduation 
should have CADD skills as well as manual 
drafting skills. 



CADD Projects for Interior Design 

Design Fundamentals Project Based on the 

Concept of Line 

Objectives. 

Interior designers must deal wit line in a 
variety of ways such as contours ji building 
details and furniture details, and as a design 
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element on fabrics, carpeting, wall coverings. 
Therefore, the objectives of this project were to 
help students observe lines in the world around 
them, to recognize the visual rhythm lines can 
create, explore the physical characteristics of 
line, to create visual rhythm with line, and to 
experience CAD as a tool in creating linear 
designs. 

Description. 

In this project, students were asked to create a 
minimum of 4 linear designs using the 
computer. The arrangement of the linear 
elements within the designs was to reflect a 
specific type of visual rhythm. This rhythm 
could be achieved by line placement, line 
repetition and/or variation in line shape, 
thickness, lightness, or darkness. Sources of 
inspiration for the designs were to come from 
either a linear translation students had made 
from a photograph or from a line drawing of a 
campus building. One of the designs was to be 
refined and matted for a formal presentation. 

Solutions. 

»re 1 Is an example of student work.( Figure 
<ne Project Solution by Amy Rohlk, 
; raiso University.) 

Summary. 

This project allowed students to become 
familiar with the hardware and software while 
exploring multiple design solutions involving 
line. When a similar project which used paper 
and pencil techniques was used, students were 
more hesitant *o explore multiple options than 
they were with the computer. Also, the design 
solutions created on the computer were more 
refined than those created with paper and 
pencil techniques. 

Floorplan-to-lBometric Projection Project. 
Objectives. 

It is often difficult for students to see the 
relationship between a 2-dimension plan and 
the 3-dimensional reality of the completed 
space. Isometric and perspective projections 
are commonly used methods of 3-dimensional 
visualization, but they are usually hand- 
drawn, time intensive and difficult for students 
to learn. It was not intended that this project 
replace learning hand-drawn perspectives. 
Instead, the objective of the project was to help 
students: 

•explore the use of 3-dimensional projections 
on a small aspect of a larger project while in 
the process of developing the 2-dimensional 
plan view and the elevations; 

•utilize the computer's ease of generating 
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multiple elevation and p spective views; 
stimulate interest in the power of the 
computer; and 

•motivate students to want to do more with the 
computer. 

Description. 

In this project students were asked to select a 
small segment of a larger floor plan which 
contained furniture, fixtures, cabinetry or 
office systems. In AutoCAD, the 3-dimensional 
elements of height and elevation for 
perspective projection and elevation views 
were entered when the plan view was created. 
The final perspectives were plotted and 
rendered off of the computer in multiple media 
depending upon student preference. In 
MicroStation they entered elements in both 
plan and elevated views while watching the 
results of what was happening in the isometric 
view since all views could be open at the same 
time. Final elevations were plotted and 
rendered in color off of the computer. 

Summary. 

In the AutoCad project, most students felt that 
the project not only helped them to leam to use 
AutoCad, but also to help them visualize the 
workstation in multiple views. (Davis, 1991) 
Anonymous student feedback in the 
MicroStation project Indicated that students 
felt that: 

•plan, elevations and isometrics were easier 

and quicker on the computer; 
•the experience gave them a better idea of how 

the space might look; and 
•problems in the plan of the space were more 

obvious in the isometric projection and since 

it was on the computer, it was easier to go back 

and modify the plan. 

Surface Design Motifs for Interior Furnishings 

and Clothing. 

Objectives. 

Interior and apparel designers are often asked 
to create custom designs for a variety of surface 
materials. The objectives of this project were 
for students to study historic and 
contemporary design motifs, to create new 
motifs representative of their own place in 
time, to apply design motifs to specific 
surfaces, and to.experience the power the 
computer has to manipulate and modify, color, 
texture, scale. 
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I'igtire I. Line Project Solution by Amy Rohlk 



Description. 

Students were asked to research historic and 
contemporary design motifs found in 
architecture, clothing, furniture and interior 
finishes (wall paper, carpet, tile). These motifs, 
along with a personal motif designed by each 
student, were then sketched on or scanned into 
the computer. Next, a variety of grids were 
developed to which the motifs were added. This 
resulted in an exploration and manipulation 
of each motif through repetition, placement, 
and variations in scale, value and color. After 
exploratory manipulations, 3 design motifs 
were chosen for refinement and formal 
presentation, and one of the three was applied 
to a surface, e.g. garment, carpet, tile, wall 
paper, or furniture upholstery. All images were 
created on the computer. Color was added to 
Laser printed copies for formal presentations. 

Solutions. 

Figures 2and 3 are examples of solutions to 
this project.. ( Figure 2. Surface Design Project 
by Lynn Barkan, Valparaiso University and 
Figure 3. Surface Design Project Application by 
Beverly Parks, Valparaiso University) 

Summary. 

Anonymous student feedback revealed that 
students felt that this project helped them to 
leam: 

more about historic design motifs; 

how to use a grid to create a repeated motif 



design; 

*how to use MicroStation; and 
*how a motif was transformed when repeated 
and applied to the surface of an object. 



APPAREL DESIGN 
Background 

The apparel design and manufacturing field is 
also experiencing computerization growth. 
According to Van De Bogart (1989) the U.S. 
apparel industry has faced severe Job losses 
over the last 10 years because of stiff 
competition from countries who can produce 
more goods for less through automated 
equipment. This industry, which Is second in 
size only to the oil industry, Is moving toward 
computerization. According to the American 
Association of Apparel Manufacturers, in a 
period of five years the Industry has gone from 
less than 5 percent computerization to the 
point where at least one quarter of the 
estimated 40,000 plus fashion designers 
working in the United States use computers. 
While this growth is strong, it still places the 
U.S. behind Europe and Japan (Van De Bogart, 
1990). 
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Figure 2. Surface Design Project by Lynn Barkan, Valparaiso University 




Figure 3. Surface Design Project Application by Beverly Parks, Valparaiso University 
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With respect to computer education for current 
design students, Van De Bogart (1989) reports 
that Van Fossen, President of Computer Design 
Inc. "...feels that seasoned designers will 
probably not change over to computerized 
designing of clothes, but that new designers 
who have grown up with computers will." (p. 30) 
Gwendolyn J. Sheldon of California State 
University, Chico feels that the age of new 
computer-literate designers is upon us and 
predicted that by 1993, "...designers entering 
the apparel industry would have to understand 
the capabilities of, and have hands-on 
experience with, computerized design, 
illustration, pattern making, grading, costing, 
and color matching equipment." (Van De 
Bogart, 1990, p. 10). 

CADD Projects for Apparel Design 

3-D Constructions with 2-D Surface Design. 

Objectives. 

The project objectives were to have students 
become aware of the importance of the 
interrelationship between the 2- and 3- 
dimensional aspects of clothing design; to 
understand the relationship between surface 
design and the 3-dimensional object; and the 
explore how th^ computer can be used to 
generate patterns for 3-dimensional 
constructions. 

Description. 

In the following projects students were asked to 
design a 2 -dimensional flat-pattern for a 
simple, 3-dimensional, geometric 
shape(square or rectangular box). They were 
then to create a 2 -dimensional design which 
would work well on the surface of the box. This 
3-dimensional box with the patterned surface 
was then to be constructed out of paper from the 
plot of the 2-dlmensional, flat pattern. 



Solutions. Solutions included the creation of a 
variety of boxes with colored surface designs on 
all sides. Since the computer generated 2-D 
patterns were to be cut, scored, and folded into 
the 3-D form, patterns were plotted on Bristol 
board with colored pens. One student evolved a 
more complicated rectangular box design using 
AutoCAD. The box pattern was then scanned 
Into the Macintosh and graphics for the box 
surfaces were designed using FreeHand. An 
AutoCAD box pattern was plotted on bristol 
board for stability and on the LaserWriter for 
the surface graphics. The two images were then 
bonded together with adhesive before the final 
cutting, scoring and folding process. 
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Expansion qf this project concept 
One student expanded the above project by 
creating a series of three dimensional shapes 
using AutoCAD, plotting these shapes on 
colored Canson drawing paper, and creating a 
3-dimensional, low relief composition 
by.scoring and folding the shapes. She then 
took the concept a step further by drawing and 
plotting simple garments using AutoCAD. 
These garments and actual fabric samples were 
then scanned into Aldus FreeHand on the 
Macintosh using AppleSCAN. The scanned 
fabric images as well as computer based 
patterns were then placed on the garment 
designs. The designs were printed on colored 
papers, cut out, scored, folded for a 3~ 
dimensional relief effect and mounted on a 
presentation board. 

Summon/. 

These projects not only helped students bridge 
the gap between 2- and 3-dimensional design, 
but they also provided them the opportunity to 
use hardware and software which would best 
fulfill their project solution needs. 

Apparel Design Using a Historic Electronic 
library of Garment Segments. 

Objectives. 

The objectives of this project were to have 
students Integrate information they learn in 
the history of clothing into the design of 
contemporary clothing; to learn how to create a 
line of garments; and to explore the use of CAD 
as a design tool in the process of designing 
garments. (In the case of one of the students, the 
main objective was to design period costumes 
for the play Hay Fever rather than a line of 
garments.) 

* 

Description. 

In this project students were asked to design a 
line of garments for today, based on garment 
designs from the past. They were to explore the 
use of CAD as a design tool In the process of 
designing this line, and compare It to 
traditional paper and pencil techniques. To 
achieve this, an electronic library of garment 
segments was created based on styles 
representative of five, 10-year periods from 
1900- 1950. (Figure 4 shows a scaled oown 
version of the library.) Line drawings of 
segments were scanned In so that bodices were 
aligned to but not attached to skirts, sleeves, 
and collars. This allowed for ease in mixing. 
Students were then able to evolve either period 
costumes using Just one time period, or new 
designs mixing components from any of the 5 
periods. For example, they could use a skirt 
from the 1900's, a bodice from the 1920's and 
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sleeves from the 1920's. They could also modify 
any of the segments should they wish to do so. 
For Instance a collar could be reshaped Into a 
bodice, or a ankle length shirt could be shorted 
into a mini length or vice versa. 

Data was collected to compare the use of the 
computer as a design tool with the traditional 
paper and pencil tools. The class assignment 
required students to design a line of six 
garment using the historic garment library as a 
reference source. Three garments were to be 
designed using standard paper and pencil 
techniques and three using CAD. The library 
was available in hard copy format for the paper 
and pencil designs, and in electronic format for 
the computer designed garments. Final 
rendering of garments was done in each 
student's own rendering style off of the 
computer. Students were also required to 
complete flats and color story boards for the 
project. Davis (1990) describes the research 
objectives and results of the study. 

Time. 

Each treatment was two instructional weeks in 
duration (12 instructional hours). The amount 
of out-of-class work varied greatly and was 
greater for the computer generated designs as 
most students were learning computer skills as 
well. A longer instructional time would have 
been preferable, but unfortunately the 
availability of the computer lab dictated the 
time frame. 



Expanded Use of the Garment Library. The 
garment library has also been used in 
connection with an interior design 
competition project where students were to 
design a resort hotel. One of the aspects of the 
project involved designing uniforms for hotel 
employees. The libraries became a invaluable 
resource, especially since interior design 
students had not had experience to apparel 
design. 

Solutions. 

Even though all students were using the same 
garment library, their design solutions were 
quite individualized and varied. The library 
provided an immediate resource of styles, it 
allowed for easy and expansive modification, 
and it allowed for a wide range of design 
solutions. The project proved to be a good way 
of integrating CAD into an apparel design 
course with students of varying computer 
experience. The hard copy library also 
appeared to increase the number of design 
alternative and variations students 
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considered before developing a garment designs 
with paper and pencil techniques. Figure 4 
shows a portion of the library and Figure 5 
shows an example of hotel uniforms designed 
using the library. (Figure 4. Historic Garment 
Library Segment created by Diane Davis and 
Figure 5. Application of the Historic Garment 
Library for Uniforms for a Resort Hotel by Gall 
Kanning, Valparaiso University.) 

Summary. 

The implications for using an historic garment 
library for the design of new and period 
garments, especially in an electronic format, is 
very exciting. The project nicely Integrates the 
use of computers into an existing course and 
provides students from diverse backgrounds 
with a guided opportunity to integrate 
historical information into the design process. 

VISUAL MERCHANDISING AND DISPLAY 

DESIGN 
Background 

Visual merchandising has been slower to 
integrate the use of CADD in merchandise 
presentation and display even though CADD is 
widely used in store design by 
interior/architectural firms designing for the 
retail market. It's use as an "in house" tool for 
merchandise presentation and display design 
has lagged behind because of the lack of CADD 
trained personnel in these areas (Davis 1985). 
This, of course, is not true for other computer 
applications in the retail business. 

Several educators are beginning to include 
compute/ designed displays, merchandising 
plans, and fixture planograms as part of their 
visual merchandising curriculum (Dilbeck, 
1990; Mehlhoff, 1990; Miller, 1990). 

CADD Projects for Visual Merchandising and 
Display Design 

Balance Through Slse Variation and 

Placement Project. 

Objectives. 

Visually strong merchandise presentation 
relies on the basic elements and principles used 
by all facets of design. Balance is one of the 
principles very critical to display design. The 
objectives of this project were to help students 
understand the principles of symmetrical and 
asymmetrical balance, to understand positive 
and negative space, to learn to use the computer 
as a design tool, and to see how efficiently the 
computer can replicate and move shapes. This 
project has also been used with interior design 
students. 



272 

256 




Figure 4. Historic Garment Library Segment Created by Diane Davis, Valparaiso University 




Figure 3. Application of the Historic Garment Library for Uniforms for a Resort Hotel by Gail 
Kanning, Valparaiso University 




Figuie 6. Symmetrical Balance Pfo)ecf by Stephanie Mochring, Purdue University 
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Description. 

In this projects, students were asked to design 
two compositions, one symmetrical and one 
asymmetrical. In the symmetrical design, one 
shape was to be created and repeated or copied 
five times for a total of six shapes. The shapes 
were scaled in size so that two were large, two 
medium and two small. These shapes were 
then arranged symmetrically within a 6"x 6"" 
square. For the asymmetrical design, one 
shape was created and scaled five times 
creating a range to size from large to small. 
These five shapes were then arranged 
asymmetrically within a 4 M x 6 ,m rectangle. 

Solutions. 

Figure 6 shows and example of a symmetrical 
balance solution for this project. (Figure 6. 
Symmetrical Balance Project by Stephanie 
Moehring, Purdue University.) The project 
proved to be a good introduction to the 
computer. It allowed for exploration in the 
development of the design shape and in the 
placement of the shapes before a student 
selected a solution. The ease and accuracy of 
scaling and duplicating the shape resulted in 
visually stronger and better crafted 
compositions than when the project was used 
with a cut paper technique. 



Display Design and Signage Development 

Project*. 

Description. 

In a visual merchandising class, students 
designed, constructed and installed displays 
for local merchants. As part of these display 
they were asked to present their design ideas to 
the instructor and the retailer before 
constructing the displays. Their display 
designs were therefore represented in a plan 
and an elevated view. These views along with 
display signage were executed on the computer. 

Objectives. 

The objectives of this project were to guide 
students in graphically working out their 
display designs; to familiarize them with the 
concepts of floor plans and elevations; to help 
them develop visual presentation skills; and to 
give them some experience using the computer 
as a design tool. 

Summary. 

Students did learn the concepts of plan and 
elevation views, and they chose to use the 
computer for creating signs rather than the 
Kroy sign making equipment because of its 
flexibility and font options.. 
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SUMMARY 

These projects have been effective in 
integrating CADD into a variety of design 
courses. This integrative approach appears to 
aid in the assimilation and retention of 
computer learning in that computer commands 
are focused on specific, meaningful tasks. 
These commands are presented and practiced 
within the design process rather than being 
presented as separate commands and functions 
with little time for manipulative practice. The 
testing of this hypothesis is an area where 
future research could be conducted. Project 
revisions and additions can be anticipated as 
students gain more CADD experience. As 
students of the future begin their college years 
with greater computer literacy, projects and 
applications will become more expansive and 
CADD will truly be a common design tool. 
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ELUCIDATING TIME MANAGEMENT ACCOUNTABILITY 
THROUGH SPREADSHEET ANALYSES 



Thomas L. Houser, The University of Tennessee, 



Abstract 

In service professions such as architecture, 
engineering anr interior design, fees are 
determined on an hourly or project basis. 
Either base challenges the professional's time 
management accountability. Firms maintain 
detailed records to vouch for efficiency and 
effectiveness and to predict, requisite staff 
commitments for future projects. 
Accountability is critical to the individual and 
the firm's success. 

The challenge for the educator lies in 
cultivating the student's sensitivity to time 
management principles. To address this 
challenge, computer-based spreadsheet 
exercises have been developed to record and 
analyze time usage. These timesheet projects 
have been used In microcomputer applications 
and business procedures classes. Students 
delineate coursework and employment 
activities for a minimum of three months, 
entering this information Into spreadsheets 
beginning midway through the study. They 
then extract and make analyses of pertinent 
data. Students also prepare tables and graphs 
by using the application's chart commands or 
by running macros developed for the project 
Representative analyses include comparisons 
of time spent on academic and non-academic 
endeavors, distribution of efforts among tasks 
associated with specific courses; patterns of 
effective time management displayed 
throughout each week and over the term; and 
peicentages of effort relegated to individual 
commitments. 

Through course evaluations, students have 
indicated these exercises to be beneficial. 
Findings reflect enhanced sensitivities to 
accountability and improved effectiveness in 
time management. 

Background 

Effective time management is important for 
everyone. ... To manage time successfully, you 
must firmly believe that every demand on your 
time is a request to give up the most valuable 
thing you possess (American Institute of 
Architects [AIA), 1988. p. 22). 

Although professionals in all fields appreciate 



the significance of effective time management, 
they often find it challenging to practice the 
principles behind it. The proliferation of 
articles on the topic highlights this point. Just 
one reference database, limited to business and 
management Journals, Indexes 494 articles on 
this topic over the past five years, nearly two 
per week (ABI /INFORM (R) Ondlsc, 1991). 

The iirst step toward understanding time 
management is knowing how one actually 
spends time. Architects, engineers and 
interior designers maintain timesheets to 
account for their time usage and to track 
progress on projects. Although used primarily 
for billing purposes, data collected from 
timesheets can be used to analyze time 
management effectiveness. 

From a business perspective, timesheets are 
very important for several reasons. Managers 
use these records to monitor the performance 
of individual staff members and of their 
departments. They employ project data as 
historic gauges while preparing contract 
proposals, first using it to anticipate pote: *1 
staff commitments, and then, to detent lie 
fees. Once a contract is secured, managers 
again refer to past records to make project 
assignments and to prepare project flow charts. 
Occasionally, they reference timesheet notes 
when Justifying invoices or while defending th* 
firm's actions. For these reasons, the 
designer's diligence in recording time directly 
affec ~ The firm's profits and losses. 
Misleading data lead to unrealistic fee 
proposals and to unachievable performance 
expectations, two unrelenting origins of 
financial fiascoes. 

A common marketing attitude places 
additional pressure on the professional's time 
management skills. In their discussion of 
ways to garner increased profits, Getz & 
Staslowski (1984) suggest that the manager 
compress the project's schedule and have the 
staff work overtime to complete it. This creates 
more "profitable projects by reducing excess 
perfection, . . . (possibly) without incurring 
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Figure 1 Weekly time allocation expectations related to one semester's curriculum. (The stacked bar 
charts represent components of course efforts. The area chart reflects the Impact each component has 
on the weelly commitments.) 



additional salary or overhead costs (p. 51). 
"Reducing excess perfection" challenges 
designers 1 professional standards, and working 
overtime, often without compensation, adds 
stress to their professional and personal lives. 
Proficient time management, even in this type 
setting, presents the designer with the 
opp. rtunity to create higher quality projects in 
less time. In this sense, designers are 
accountable to themselves to develop efficient 
time usage patterns. 

The expanded use of microcomputers provides 
another rationale for this project. Crosley 
notes, "the thinker [is becoming] the doer," and 
professionals now perform many of the tasks 
that secretaries finished in the past (1989, p. 
109). Sign'ilcantty, a recent AIA report reveals 
that financial management software, used In 
part for time record management, is the third 
mpst popular type of computer applications 
program used in architecturul firms preceded 
by vvrd processing and speciiication-writing 
software, and followed by coroputer-rided 
design programs (Witte, 1990, p. 113). Some 
applications recommended for smaller firms, 
e.g. Timeslips III, track the duration of 
activities as they are being performed. The 
programs then use this data to generate 
invoices (Tully, 1990). The designer is the 
catalyst for the entire time record keeping 
process. As a credential, computer literacy is 
more Important than ever. 

Students experience significant demands on 



their time management abilities, too. They 
must balance curricular, extracurricular, 
social, and, often, employment activities. The 
magnitude of this challenge can be illuminated 
through an examination of course 
requirements and professorial expectations. 
During one semester at the University of 
Tennessee, interior design students take 16 
hours of coursework, spend 26 hours a week in 
class, and are expected to commit 27 to 39.5 
hours on outside preparation (see Figure 1). For 
16 hours of credit, students, ideally, consume 
from 53 to 65.5 hours of their time each week 
(Houser, 1991). 

The students perception of self-Initiated time 
management is a critical factor in controlling 
anxiety. After a recent study of correlations 
between academic performance and stress, 
Dipboye and Phillips (1990) reported that 
"students who perceived (effective self-]control 
of their time reported significantly greater 
evaluations of their performance, greater work 
and life satisfaction. . . and fewer Job induced 
and somatic tensions" (p. 760). 

Timesheet Project. A computer-based 
spreadsheet project can address all these 
challenges. Students in microcomputer 
applications and business practices courses use 
the timesheet exercises described here. 
Procedures are modified to reflect course 
objectives. In microcomputer classes more 
emphasis is placed on the spreadsheet; in 
business practices more is placed on time 
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management. 

The objectives behind this timesheet project 
are (a) to develop procedures for analyzing time 
usage, (b) to cultivate the student's sensitivity to 
time management accountability, (c) to foster 
computer literacy, and (d) to create an extensive 
database upon which to conduct related 
research. 

Students delineate coursework, employment 
and miscellaneous activities for three months. 
They begin maintaining timesheets during the 
second week of the semester, after 
comprehensive time management discussions 
during the first. Midway through the project 
students begin entering information into 
computer spreadsheets. With this data as 
tangible evidence, they analyze their time 
usage from multiple quantitative perspectives. 
They also examine their records in light of 
intangible time management concepts, e.g., 
intent vs. reality, planning vs. reacting, and 
organizing vs. drifting. The last two weeks of 
the semester are not recorded to allow the 
students to complete their projects. 

The validity of self-reported data is critical. 
Student integrity is nurtured through an 
atmosphere of trust among the students and the 
instructor. They are reminded regularly that 
the underlying objective of the project is for 
them to understand how they use their time. 
This cannot be accomplished without accurate 
records. At no time should students feel 
threatened by conclusions drawn from their 
data; nor should they feel elation from 
receiving external praise. Both threats and 
kudos can encourage the submission of 
dishonest information. Therefore, students 
are neither rewarded, nor punished for 
spreadsheet findings. 

Project grades are based on students' record 
keeping abilities: not on how they spend time, 
but on how effectively they record and analyze 
it. As the study proceeds they are encouraged to 
discover and address emerging time usage 
patterns, but their project grades are not 
contingent upon demonstrating improved time 
management skills. Accountability is the issue 
here. 

Although timesheets are collected weekly, they 
are checked during other class periods and 
during chance encounters outside class. 
Frequent examination engenders accuracy by 
ensuring records are current and by reducing 
post facto guesswork by procrastinators (see 
AIACh. 1.11, p. 18). 



Since they know their projects are part of a 
larger study, students are assured of anonymity 
in ensuing research procedures and reports. 
This non-disclosure policy helps counter 
vested interests that may encourage the 
reporting of specious information. 

Project Methodology 

Students maintain time records In manners 
similar to those employed by professionals 
(see, for example, Getz, 1984, p. 128). The 
students: 

♦identify projects by name and number; 
♦indicate generic tasks and activities; 
♦make germane observations on specific 
activities; and 

♦tally the duration of each project effort. 

Further, they log activities based on specific 
ground rules adapted from accepted 
professional procedures (see Figure 2). 

Using Timesheet Formats. Students use two 
timesheet formats: hourly and weekly (see 
Figure 3). The first tracks activities throughout 
each day and the second summarizes them by 
the week. Students submit both timesheets for 
review and comments, before entering data 
into the computer. 

A chronological timesheet is an efficient 
instrument for daily record keeping. Because 
of the erratic nature of students' schedules, 
each form is subdivided into 15 minute 
segments around the clock. The student makes 
a new entry each time a task is begun and then 
notes its duration. After the day's activities are 
complete, the student indicates subtotals for 
each prqject:task combination. 

Students use the second timesheet format to 
summarize all activities for the week. Each 
row of the timesheet represents daily subtotals 
for one type of task on a specific project. 
Students transfer the information entered onto 
this form into the computer spreadsheet's 
database, after it has been edited for 
inconsistencies with their daily timesheets. 

Entering the Spreadsheet Database. At this 
point each student starts entering records into 
a preestablished database, the part of a 
spreadsheet that contains the data to be 
analyzed later. The database for this project 
Includes the 12 fields discussed below. 
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1. 



2. 



3. 

4. 
5. 



8, 
9. 



Maintaining time records requires self-discipline. Keep your timesheets with you as you go through 

your day. If you make an entry each time your activities change, your records will be accurcta. It is 

difficult to be precise if you wait until the end of the day to make entries. 

Record time In 15 minute intervals. This will require some judicious decision-making, as few 

activities start and stop on the quarter hour. In general, round an extra five minutes down, and ten 

up. To keep from under- or over-crediting routine activities, reverse this process periodically. The 

approach you take is assumed ta be relatively consistent throughout the study. 

Use decimals to record partial hours: record 15, 30 and 45 minutes as .25, .50, and .75, 

respectively. 

Do not record time spent in transit between home and the university or your place of employment. 
Once the first activity occurs at the university, record time spent traveling to the next with the 
latter. For example, record the 10 minutes it takes to cross campus to a SO minute locture with that 
class' time: log one hour for the course. This is consistent with the professional practice of charging 
travel time to a specific design project. * 
Only record SOCIAL time between 6:00 a.m. and 5:00 p.m., Monday through Friday, excluding time for 
meals and personal errands, You will note the effects interruptions have on the productive parts of 
your day. Professionals account for every minute during the work day. Assignments not completed 
by "quitting time* must be finished on overtime. 

ILLNESS, EMERGENCY and HOLIDAY are special project categories. Entries for these can occur only on 
a weekday between 6:00 a.m. and 5:00 p.m. and must equal eight hours or fewer. When productive 
entries occur on the same day, record the difference between the productive activities and eight 
hours. For example, if you work on a paper from 7:30 to 9:45 p.m. on Tuesday during spring break, 
make a 2-1/4 hour entry for the class and record a 5-3/4 one under HOLIDAY. If coursework or 
employment activities last longer than eight hours, no entries are made under these three categories. 
HOLIDAY entries preempt ILLNESS, EMERGENCY and SOCIAL ones. 

If you have a job in which you are allowed to study while "on the clock' subtract productive 
coursework efforts from your EMPLOYMENT entries. This is comparable to the professional practice 
of charging indirect time to a specific project. 



Figure 2 Ground rules established for students to record information. 



Week: Weeks are numbered to simplify 
extracting information from databases. 
Classroom experience has shown that students 
find It difficult to extract information based on 
dates. 

Project Identification. This coding system 
provides thebase for organising data. Course 
designations consist of three letters and three 
numbers. (Note the use of three letters Is 
critical: It prevents the software from 
confusing Project IDs with cell references.) One 
word Identifiers are used for EMPLOYMENT, 
ILLNESS, EMERGENCY and SOCIAL activities. 
Subnumbers divide a project identification 
into components. For example in a third-year 
studio class, IDE340.00 encompasses general 
activities, IDE340.01 represents the first 
project, and IDE340.02 the second. Likewise, 
EMPLOY.Ol may refer to a work-study 
assignment and EMPLOY. 02 to a part-time Job 
at a local design firm. Consistent use of 
identifiers is critical, as Project IDs are a 
primary criteria for organizing and extracting 
spreadsheet information. 



Project/class: This text field identifies the 
project or the subproject. Entries may be as 
broad as the course name, for example, Art 
History; or as specific as a project name, such 
as Giotto paper. This information serves as a 
check for the accuracy of the entire project 
record. 

Task: The task coding system identifies 
generic project activities. This provides a 
means for comparisons of similar activities 
among course^ or projects. The validity of 
subsequent database analyses is contingent 
upon the consistent use of task designations by 
all students. 

M, TU, W, TH, F, SA, and SU: Each database 
record contains the time spent throughout the 
week on a particular prqject:task combination. 
Subtotals for each day are indicated in the 
appropriate fields. Students are cautioned to 
leave cells blank If no activity occurs on a 
particular day. (A zero in a cell distorts certain 
spreadsheet functions, like Counting or 
Averaging field entries.) 
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Figure 3 Tlmesheet formats. (Students maintain dally tlmesheet Information In a 5"x9" binder, and 
they enter weekly data into the computer spreadsheet.) 



DIRECT TIME 



INDIRECT TIME 



DESIGN TASKS 
D.1 Freehand work 
Drafting 
Programming 



D.2 
D.3 



ACADEMIC TASKS 
A.1 In class (lecture) 
A. 2 Studying new material 
A. 3 Reviewing material 



EMPLOYMENT 
W.1 On-campus 
W.2 Off campus 



GENERAL TASKS 
G.1 General 
1.1 Illness 
E.1 Emergency 
S.1 Social 



Figure 4 Primary task designations u&d In this study. 
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Actdtmlc: Non-Actdtmlc Efforts 



Holiday: 12.3% (4 h/s.) 



Dance: 6.2% (2 hrs.) 



CAD: 9.2% (3 hrs.) 



ID Business: 
7.7% (2.5 hrs) 



Biology: 12.3% (4 hrs.) 




Human Development: 
9.2% (3 hrs.) 



ID Studio: 43.1% (14 hrs.) 



Weekly Ptrctnttgtt 



Figure 5 Representative time usage charts based on weekly extracts. (Hie bar charts depict one 
student's activities during the fourth week of the project; and the pie chart shows activities during the 
second week.) 



Totals: Because ol the presence of preset 
formulas, the weekly subtotal for each 
projectrtask combination appears hen* 
automatically. In addition to these fields, the 
spreadsheet contains a Remarks column next 
to the database. Students may use this area to 
explain project activities. Although few 
students enter comments into this field, most 
do make observations in the corresponding 
column on their hand-written timesheets. 
These notes do not affect computer analyses. 

Use of the Task field on this project merits 
special consideration. In the design 
professions this information may be 
subdivided into entries for the project's design 
phase, e.g. schematics or design development, 
and for the specific activities, like drafting or 
space planning. The complexity level 
employed in coding activities varies 
significantly among firms. One representative 
sample identifies 33 design phase and task 
combinations and eight indirect expense 
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categories (Getz, 1984, p. 129). 

Few undergraduate students completely 
understand the nuances Inherent in such a 
system. After experimenting with the number 
and type of task designations, 12 have been 
adopted for the more recent studies: three each 
for design studios and academic courses; two 
for employment; and four for personal matters 
(see Figure 4). Experience has shown this level 
of detail approximates the understanding of 
task categories held by most second and third- 
year students. Gren+er delineation baffles the 
younger students, anc less nets insufficient 
data to study work patterns. Although students 
may add to this list with prior approval, few 
second, but most third-year students exercise 
this option. 
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Figure 6 Representative semester-wide time usage charts, (The area chart reveals dramatic, time 
commitment fluctuations for one course; and the bar chart reflects disproportionate relationships 
between course credits and the student's efforts. These observations must be qualified. One cannot 
look just at numbers and conclude the student is spending too much time on one course; or, conversely, 
conclude that the course demands too much from the student.) 



The Macintosh 
Environment 



desktop hierarchy 
opening documents 
menus 
windows 

scrolling procedures 
using the Clipboard 
saving documents 



Spreadsheet Terms 
and Concoptft 

• cells 

• columns and fields 

• rows and records 

• named ranges 

• database 

• criteria 

• criteria ranges 

• extract 

• output ranges 

• text and numbers 

• formulas 



Fulllmpact 
Procedure 



selecting cells 
entering data 
adding formulas 
entering cell references 
editing cell contents 
inserting rows 
sorting databases 
using icon bars 
running macros 
creating views 
charting data 
editing charts 



F.'gure 7 Basic software terminology and procedures students need to understand to complete 
tlmesheet projects. 
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By modifying the task coding system, students, 
educators, or professionals in other disciplines 
can use this tlmesheet project effectively. The 
degree of detail recorded must match both the 
level of planning the Individual is able to 
achieve and the hierarchy of information that 
the manager or researcher can extract (see AIA, 
Ch. 1.11, p. 18). 

Extracting Data and Making Reports. Learning 
how to record time is important, but the 
significance of this project is analyzing how 
students use their time. Here is where the 
power of a computer spreadsheet shines. Here, 
too, is the point at which one can turn the 
routine task of keeping timesheets into an 
illuminating catalyst for self-initiated 
professional development. Visually oriented 
design students find the charts prepared In this 
part of the project to be both appealing and 
helpful. Almost without exception, skeptical 
individuals see the value of this projects as 
soon as the first graph appears on the screen. 

Students extract data by weeks, days, project 
identifications, tasks, durations of time spent, 
and combinations of these criteria. They then 
prepare graphs to analyze data and to report 
their findings. Both weekly and semester-wide 
efforts are examined (see representative charts 
in Figures 5 and 6). 

Weekly analyses include computing project 
subtotals; comparing time spent on academic 
and non-academic endeavors; evaluating 
percentages of effort among courses; and 
comparing task efforts associated with specific 
courses. These weekly analyses provide 
snapshots of students' efforts. They see the 
distribution of their efforts among courses, the 
depth to which they are applying themselves, 
and the balance they are striking among 
academic and non-academic activities. They 
can then use these analyses as a base to address 
deficiencies in their time management 
practices. 

Semester-wide analyses produce more 
meaningful information than weekly ones. 
Most are similar to their briefer counterparts; 
however, temporary time warps -- caused by 
factors like tests, due dates, football games and 
illnesses - are averaged into the bigger picture. 
Students gain a truer perspective of their 
efforts, as these analyses portray trends, as 
well as data. 

Several semester-wide studies deserve 
additional attention. The first tracks the 
student's weekly academic efforts throughout 
the semester. Data linkages connect weekly 
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extracts to a secondary database. Because of 
these linkages, the software automatically 
updates the database each time the student adds 
a weekly extract to the project. An existing 
area chart, linked to this secondary database, 
graphically depicts shifting emphases among 
courses. When reviewing their charts, students 
can evaluate how working ahead on particular 
projects could have affected the intensity of 
their out-of-class commitments throughout the 
term. 

Another significant semester-long analysis 
examines the percentages of the student's 
academic commitments among courses. Data 
linkages to weekly extracts, modified by 
spreadsheet formulas, create another 
secondary database. This consists of three 
fields: The first indicates the percentage of the 
student's courseload each course represents; the 
second reflects the percentage of the student's 
total academic effort each course received; and 
the third factors the commitment percentage 
against the course credit percentage. The 
information is placed automatically into a 
horizontal bar chart for comparisons. Like the 
weekly analysis described above, this chart is 
updated each time the student adds a weekly 
extract. When reviewing this information, 
students see how their time allocation 
decisions impact all of their courses. They can 
then determine if their priorities are well- 
placed. 

Computer Techniques. Students prepare their 
projects on Apple Macintosh SEs and later 
models, using Ashton Tate's Fulllmpact 1.0 and 
2.0 spreadsheet programs. Some students 
enhance their report graphics via Claris 1 
MacDraw II. Note, however, the procedures 
discussed here can be adapted for different 
spreadsheet programs on Macintosh and other 
computer platforms. 

In microcomputer courses, students go through 
Fulllmpact's computer-based tutorials, and 
they complete smaller spreadsheet projects 
before entering data on this one. In 
professional procedures courses, students learn 
a minlmuri of basic spreadsheet terms and 
operations before they input data from their 
timesheets (see Figure 7). This enables them to 
enter information and to see results without 
becoming overwhelmed by either the computer 
or the software. 
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^ifiure 8 Macros and custom icon bars developed for tlmesheet project. (Students can switch among 
these two command bars and Fullimpacfs standard bars. All but four of these icons launch macros 
developed for this project. One of the two bars is dedicated to the project's 13 weekly macros.) 
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Just as students are accountable for how they 
spend time, so are Instructors accountable for 
how they ask them to spend It. For this reason, 
students follow one of three sets of procedures 
for extracting and analyzing database 
Information. Students use Exercise A In 
microcomputer applications courses. Those In 
business procedures use Exercises B and C. 
These two variations reduce the time it takes to 
complete the project, and thereby, help to meld 
course and project objectives. (For additional 
observations about these exercises, see 
Computer Techniques in the Findings section.) 

Exercise A: Students input records into an 
existing document having a preestablished 
. database range. From this point they prepare 
all criteria ranges, extracts, charts and reports. 
Exercise B: As in the first exercise, students 
input records Into a document with an existing 
database range. 

After their records have been reviewed and 
sorted, they transfer their databases into 
another document. (In some programs, they 
could simply switch to new worksheets at this 
point.) This second document is organized into 
weekly and semester-wide views. Each view 
contains all criteria ranges, references, and 
formulas needed to complete relevant analyses. 
Students extract information at designated 
output range locations. They then paste 
Preformatted charts onto their extracts. After 
editing charts and making observations, the 
students are finished. 

Exercise C: Students follow the procedures 
prescribed in Exercise B to establish their 
databases. They then run macros that 
automatically complete weekly and semester- 
wide time usage analyses. After editing the 
spreadsheets and making observations, the 
students are finished with the project. 

Project Macros. Stated simply, a macro is a 
series of prerecorded commands, launched by a 
single click of the mouse. The nineteen macros 
I developed for this project are described below. 
All of these can be accessed through 
Fulllmpacfs Macro menu, and most can be 
launched from one of the two custom icon bars, 
also developed for th;s project (see Figure 8). 

Insert Data Elsewhere macro: 
This global macro expedites the process of 
copying the values of cells containing cell 
references and./or formulas and entering them 
into another part of the spreadsheet. This 
macro perfonns Fulllmpacfs two-part, Paste 
Special command. Because the use of the 



standard Paste command can make cell 
references and formulas meaningless, this 
macro replaces formulas with their values 
before entering data in the new location. 

Week 1-13 macros: These local macros: 
♦extract Information from the database 
according to multiple, prescribed criteria 
ranges; 

♦organize that data into tables and charts; 
and 

♦place appropriate data into a predetermined 
report format. 

Percentage of Effort and Chart All macros: 
These project macros extract data from the 
output ranges created by the weekly macros and 
prepares the semester-wide analyses described 
earlier in the discussion on student reports. 
• Sort Data macro: This local macro eliminates 
the data scrambling that occurs if students use 
the Sort command when their records contain 
references to cells In other rows. This macro: 

♦copies the selected data range; 

♦replaces formulas ?md cell references with 

their values; 

♦pastes the modified data on top of the 
original; 

♦sorts the records; and 

♦reinserts formulas in the line total field. 

Document Setup macro: 
This macro prepares a new document for this 
project by setting all page layout parameters, 
including identifying the course and student. 
The macro is used once on each project. 

Switch Icon Bar macro: 

Because Fulllmpact only allows one custom 
icon bar at a time, this macro toggies between 
the two developed for this project. Both custom 
bars contain an icon for this macro. Students 
can switch between the two with one click of 
the mouse, without remembering the steps for 
loading a new bar. 

All Weeks macro: 

The longest macro developed for this project 
merges databases from all timesheets and runs 
all other macros. I use this macro for grading 
and research purposes. Because students do not 
need this macro, it is not Included in their 
documents 1 Macro menu or icon bars. 

Findings 

Accuracy of Self-Reported Data. Evidence of 
record keeping accuracy exists in the fact that 
students willingly reported lower levels of 
outside activities for my classes than I 
reasonably could expect, based on assignments 
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and published expectations. Theoretically, had 
students felt vested interests tn 'looking good" 
to me, activities in my courses would have been 
exaggerated. 

For several other courses, students 
consistently reported semester-wide efforts not 
much greater than the total class times. In fact, 
records for one course often showed that 
students spent less than 32 hours on it over a 13 
week period: That class was in session 39 
hours! ("My grades went up after I quit going to 
class," observed one student.) 

Additional evidence of accuracy was garnered 
by: 

♦comparing records among students enrolled 

in the same course sections; 
•verifying significant course assignments, 

deadlines and exam dates; 
♦checking time usage patterns for 

corresponding increases and decreases 

among courses; and 
♦checking class attendance records. 

No significant evidence of implausible 
timesheet entries existed within the 
spreadsheets of 56 of the 63 students involved 
with these projects. Consequently, 89% of the 
records submitted can be data sources for future 
studies. 

Hand-written Timesheet Formats. 
During the first year of the study, students 
maintained all records on hand-written copies 
of the weekly form. Some expressed difficulty 
in tracking which row reflected particular 
combinations of projects and tasks. The use of 
dally tlmesheets alleviated this problem, 
although students had to transfer information 
to the weekly format before entering data into 
their spreadsheets. When this procedure was 
followed, more students expressed confidence 
in the accuracy of their databases. 

From a spreadsheet perspective, the weekly 
format has distinct advantages over the daily 
one. First, it requires fewer entries per week; 
thus the student can input data more quickly 
and with fewer chances to make mistakes. 
Second, this format consumes less computer 
memory, as both the database and the 
subsequent extracts are shorter. Third, because 
many spreadsheet programs recalculate 
information and regenerate screens slowly, 
using this format is good time management in 
itself! 

Additional Database Field. 

A thirteenth database field, similar to the 



billing column on most design firms' 
tlmesheets, should be added. In professional 
practice, such a column indicates if a time 
entry is: (a) billed directly to the client; (b) 
recorded on the project, but not billed directly 
to the client; or (c) not billed to the client or 
charged to a project, but noted as a general 
activity associated with the operation of the 
firm (e.g. general meeting, space 
reorganization, Illness, or holiday). These 
classifications commonly are referred to as 
billable, record, and non-billable, respectively. 
(Individual design firms may have additional 
guidelines for using these classifications.) 

For student projects this additional field would 
be named charge classification, and would 
indicate if a time entry is: (a) billed to an 
employer; (b) recorded for the course for 
attending class or going on a field trip; (c) 
credited to the course for working outside of 
class; or (d) not charged, reported or billed to a 
course or employer, but noted as a general 
student activity (e.g. sorority or fraternity 
meeting, emergency, illness, or holiday). To 
acclimate students to professional Jargon, 
these classifications could be referred to as 
billable, record, credit, and non-billable, 
respectively. 

Adding this field would provide several 
significant benefits. It would: 

♦simplify some extractions and macros. 

♦resolve the Issue of how to identify the use of 
time spent in studio classes (Is it lecture time, 
or is it project time? In theory it is one or the 
other: In reality it can fluctuate between the 
two, depending on the instructor, the 
students, and the project.). 

♦emphasize the amounts of time, or lack 
thereof, students spend on coursework 
outside of class. 

Computer Techniques. 

Students gain valuable spreadsheet experiences 
by completing Exercise A; however, the project 
consumes an inordinate amount of total course 
time for both the students and the instructor. 

Although students complete their projects in 
much less time by following the procedures set 
in Exercise B, they gain less spreadsheet 
experience. This approach is more appropriate 
for business courses than for computer courses, 
unless students develop additional criteria, 
extracts, and charts. Moreover, these 
procedures produce very large spreadsheet 
documents, in the range of 400,000 - 500.000 
bytes each. 
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By using the macros In Exercise C, students 
complete their projects in less time, with less 
effort, and with fewer errors, than by following 
the either of the preceding methods. Like 
Exercise B 9 this approach is more appropriate 
for non-computer courses. However, the 
resulting documents are smaller and contain 
fewer errors. This is also the most efficient of 
the three exercises for research purposes. 

Student Errors. 

Working with students on a preestablished 
spreadsheet is somewhat like playing Russian 
Roulette on the computer: It's difficult to know 
what they're going to throw out next! 

Experience has shown their more common 

data entry errors include: 
♦using Inconsistent text entries (project name, 
spelling, abbreviations, extra spaces) 
♦omitting information repeated in 
consecutive records; 

♦misadding daily subtotals for specific 

prq)ect:task combinations; 
♦using incorrect decimals, e.g. 45 minutes 

entered as .45, instead of as .75; and 
♦entering project identification numbers like 

cell references, e.g. ID360 instead of IDE360. 

Likewise, their more common command errors 
include: 

♦copying and pasting cell references or 

formulas instead of values; 
♦pasting data into cells that already contain 

data; 

♦sorting records containing references to cells 
in other rows; 

♦adding or deleting rows or columns that 
affect data in other parts of the spreadsheet; 
and 

♦not re-extracting data after they have made 
changes to their databas f? . 

As is the case with most computer novices, 
their most common procedural errors are: 
♦not saving their documents frequently; and 
♦not making backup copies. 

Fortunately, the majority of these errors can be 
corrected through editing. Others can be 
avoided by following common tips on 
spreadsheet organization, e.g. working 
diagonally on the spreadsheet when 
establishing smaller databases and extracts. 
The most fatal error occurs if students use the 
Sort command inappropriately. 
Unfortunately, detecting this error is difficult; 
and it is impossible to correct once another 
operation is performed. Using the project's 
Sort Data macro averts this disaster. In worst 
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case scenarios students have to revert to their 
most recently saved versions. 

Student Feedback. Although some students 
complain about keeping timesheets for an 
entire semester, most realize they will follow 
similar procedures throughout their careers. 
Very few students make negative comments 
about the project on anonymous course 
evaluations; even then, students most often 
follow complaints with positive statements on 
the value of the project. 

The following comments are relatively typical. 
One student wrote,'! find myself getting mad at 
myself if I lose half an hour because I'm 
disorganized." One married student told a 
graduate assistant, "IVe gotten to a point where 
I feel guilty if I waste time. I feel like I'm getting 
more done at home and school." Several weeks 
after the end of one project, a fairly well- 
organized student said, "I really enjoyed seeing 
where my time was going. I didn't like what I 
saw at first, but I was glad to know." Perhaps 
the most unnerving comment came from a 
student who wrote, "I wish all our instructors 
could see our charts, maybe they would plan 
their courses better, so that (course)work was 
spread more evenly across the semester. If [he 
or shel were better organized, I could be, too." 

Future Research 

Having developed this computer-based 
Instrument, I can now begin analyzing data 
from the students' databases to determine if 
there are correlations among factors like 
patterns of time usage, efforts dedicated to 
specific tasks, course grades, grade point 
averages, and characteristics of various 
student population groups. The Impact time 
management instruction has had on 
participating students can be examined by 
comparing xademic records of participants to 
non-participants. Finally, significant area of 
study exists in the comparison of the students' 
self-reported patterns of time usage and the 
expectations of their instructors. 
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MEDICAL HYPERTEXT BASED UPON COGNITIVE FLEXIBILITY THEORY 



David H. Jonassen. University of Colorado at Denver 



Advanced Knowledge Acquisition 

In order to solve complex, domain- or context- 
dependent problems, learners must acquire 
advanced knowledge, which occurs after the 
Introduction of knowledge and prior to the 
development of expertise (Splro et al, 1988). 
Since Introductory learning stresses simple 
reproduction of rigid examples based upon 
limited cases, learners often fall to achieve 
advanced knowledge. Without being able to 
reason with or apply the information that they 
acquire, learners will not be able to transfer 
the knowledge to novel situations. Since 
nearly every case In medicine Is unique, 
transfer skills are essential. Learners need 
Instructional conditions that stress multiple 
Interconnectedness and different perspectives 
on the same cases. Learners need flexible 
representations of domain knowledge. Hence, 
we recommend cognitive flexibility theory. 

Solution: Cognitive Flexibility Theory 
and Hypertext 

Cognitive Flexibility Theory 

Cognitive flexibility theory is a conceptual 
model for Instruction that is based upon 
cognitive learning theory. Its intention is to 
facilitate the advanced acquisition of 
knowledge to serve as the basis for expertise In 
complex and ill-structured knowledge 
domains. The primary tenets of cognitive 
flexibility theoiy (Splro et al 1988) include: 

Avoids oversimplifying instruction. 

Ill-structured knowledge is particularly 
prevalent in biomedical education. Because of 
the interdependent nature of physiological sub- 
systems, environmental constraints, and their 
medical problems, biomedical eduction should 
stress conceptual Interrelatedness and 
combinations. Instruction should reflect the 
complexity that faces practitioners, rather 
than treating medical problems as simple, 
linear decision making. 

Provides multiple representations of content. 

Among the weakest assumptions of most 
learning theories is that there Is a best way to 
conceive knowledge, that is, there is a single 
schema or concept that best describes any 
object or event. Tliis assumption relies on an 



objective reality as the one true representation. 
This objectivistic assumption is gradually 
being replaced in learning psychology with a 
more constructivistic assumption that claims 
that we each individually construct meaning ( 
in the form of schemata) for objects or events 
based upon our experiences that we relate to 
them (Jcnassen, in press). In order to 
comprehend the complexity of the world, we 
must perceive and reconcile its different 
Interpretations. Transfer of acquired 
knowledge to novel situations, which is 
essential in problem solving, requires multiple 
mental representations that are best achieved 
through the instructional use of multiple 
analogies (Gick & Holyoak, 1983). "It is only 
through the use of multiple schemata, concepts, 
and thematic perspectives that the multi- 
faceted nature of the content area can be 
represented and appreciated" (Jacobson, 1990, 
p. 21). 

Emphasizes case-based instruction. 

Rather than basing instruction on a single 
example or case, it is Important that a variety 
of cases be used to illustrate the content 
domain. The more variegated these cases are, 
the broader the conceptual basis that they are 
likely to support. The ill-structuredness of any 
knowledge domain is best illustrated by 
multiple perspectives or themes that are 
inherent in many cases. The extensive use of 
multiple cases also supports a variety of 
clinical contexts for the acquisition of 
biomedical knowledge. 

Context dependent knowledge. 

The importance of a clinical context for the 
acquisition of biomedical knowledge cannot be 
overemphasized. The basic sciences approach 
to instruction decontextuallzes knowledge. 
Information is presented and knowledge 
acquisition evaluated either in a contextual 
vacuum or in a situation that is dissimilar to 
its ultimate use. At some future date, medical 
students are required to retrieve that 
information and try to relate it to its new 
context. However, their schema may likely be 
too constrained to support that type of 
reasoning. Biomedical knowledge is best 
acquired in relevant situations that are likely 
to be encountered by the student as a 
practitioner (Brown, Collins & Duguid, 1989). 



Knowledge construction; not transmission. 

Rather than transmitting objective knowledge, 
learners must become responsible for 
constructing meaningful knowledge 
representations in order to adapt and use it in 
novel situations. The basic sciences approach 
to instruction stresses recognition and recall of 
specific information, which makes it less 
useful. In order for knowledge to become 
transferable to different problems, it must be 
constructed using parts from different cases 
and contexts. 

Supports complexity. 

It is important that medical practitioners 
acquire non-compartmentalized knowledge 
that are not based upon rigid, external 
knowledge structures. Rather than mapping 
knowledge onto the learner, the learner must 
map different contexts onto his/her own 
knowledge as it is being acquired In order to 
support the transferability of that knowledge. 
This is best supported, according to cognitive 
flexibility theory, by the use of multiple 
representations and different thematic 
perspectives on different cases. In order to 
construct useful knowledge structures, learners 
need to compare and contrast the similarities 
and differences between cases. 

Effectiveness of Cognitive Flexibility Theory 

Although flexibility theory is new, it has been 
subjected to empirical verification. Flexibility 
theory has been operationallzed in terms of 
case-based instruction that requires learners 
to examine information from a variety of 
perspectives (ie. criss-crossing the landscape). 
Research on flexibility theory has 
hypothesized that control group learners 
studying traditional, single-perspective 
materials would concentrate more on detail 
and memorization, and that learners studying 
flexibility theory materials would be able to 
transfer learning better. Spiro et al (1987) 
found that indeed control group learners 
outperformed the experimental group on 
reproductive memory tests, but the flexibility 
theory learners exceeded controls on six 
different measures of transfer. Hartman and 
Spiro (1989) proposed a new form of text based 
upon post-structuralist thought and flexibility 
theory which Included multiple perspectives, 
flexible representations, and assessment of 
transfer. They found no differences In 
reproductive memory, but significant 
differences in transfer and knowledge 
application favored the flexibility group. 
Finally, Jacobson (1990) compared a computer- 
based drill treatment with a criss-crossing 



treatment which provided multiple 
representations of knowledge, linked abstract 
ideas to case examples, and stressed the 
interrelationships between surface and 
structural knowledge components. The drill 
group recalled more facts than the criss- 
crossing group, but the latter attained higher 
scores on all knowledge transfer tasks. The 
research has provided consistent empirical 
support for the theoretical predictions of 
cognitive flexibility theory. Flexibility 
materials engage learners in more meaningful, 
transfer-oriented, advanced knowledge 
acquisition. Wi at then is the best medium for 
presenting infoi nation organized by cognitive 
flexibility theory? 

Cognitive Flexibility and Hypertext 

Perhaps the ideal medium for conveying 
cognitive flexibility theory is hypertext (Spiro 
& Jehng, 1990). Hypertext is non-linear or 
dynamic text. Although its antecedents date to 
1945, the term hypertext was coined by 
Theodor Nelson (1974, 1978, 1981) to describe 
non-sequential writing. In traditional text, the 
readers /learners are expected to follow the 
author's style and organization of text, which 
reflects the author's knowledge structure. 
Hypertext, on the other hand, allows the user 
immediate access to any piece of text or 
information in the knowledge base. In 
hypertext, readers are not constrained by the 
subject matter structure or by the author's 
organization of the text. Since an individual's 
knowledge structure is unique, the ways that 
individuals prefer to access, interact with, and 
Interrelate information is also distinct. So, 
access to and organization of information 
should be under the control of the learner. In 
hypertext, users may explore information and 
even alter it in ways that make the 
information more comprehensible. 

Hypertexts possess some or all of these 

characteristics (Jonassen, 1989): 
•Nodes or fragments of Information 
•Associative links between the nodes 
•Network of ideas formed by the link structure 
•Organizational structure that describes the 
network 

•Ability to represent explicitly the structure of 
information to the structure of the hypertext 
•Dynamic control of Information by the user 
•High level of interactivity with the user 
•Database-like structure for storing 
information 

•Multi-media information environment 
(hypermedia) 

•Multi-user access to the information. 
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Implementation: Transfusion Medicine 
Hypertext 

Content 

Transfusion medicine includes those 
disciplines in medicine which are involved 
with the collection, processing, and storing of 
cellular and acellular components of blood, the 
administration of blood components and 
products, ar J the survival and function of the 
elements in transfusions. The field is very 
broad and entails a number of basic sciences, 
including biochemistry, immunology, 
physiology, and clinical areas, such as 
hematology, surgery, Internal medicine, and 
pediatrics. 

Learning Context 

This Cognitive Flexibility Hypertext on 
transfusion Medicine is being developed as 
part of a Transfusion Medicine Grant from the 
National Institutes of Health. In the past, 
eduction of medical students, residents, and 
practicing physicians in the area of blood 
banking and hematherapy has been 
Incomplete and plagued with misinformation 
and opinion. As transfusion medicine has 
grown more complex, the need for additional 
training has become obvio The purpose of 
the grant is to identify ^ad Implement a 
transfusion medicine curriculum which draws 
on multidlsciplinary contributions, some of 
which is to be delivered via computer-based 
learning. Through the educational activities of 
this project, outstanding students and 
clinicians will hopefully be drawn to this area. 
Ultimatelv, the project hopes to enhance the 
quality ol medical care received by patients in 
the region. 

Knowledge Domain 

An analysis of the relevant outcomes of 
Instruction for third year medical students and 
residents related to transfusion medicine 
determined that the primary goal of 
transfusion medicine for physicians is risk 
assessment. This knowledge base comprises 
the declarative knowledge base of transfusion 
medicine, that is, the information relative to 
risk events in transfusion medicine that 
physicians need to acquire. The database 
describes the different aspects of each event, 
such as the pathophysiology, symptoms, 
screening tests, treatments for reactions, and 
so on. That is, causes of these events may 
produce a variety of possible symptoms or 
require a variety of optional medical 
interventions. These are clear Indicators that 
the transfusion medicine knowledge base is ill- 
structured. Because of its ambiguity, no single 



case could represent all of the possible 
transfusion effects. That is, there is no 
prototypic case in transfusion medicine 
because the causes and effects are interactive 
and depend upon whose perspective the case Is 
being viewed from, the condition or physiology 
of the patient, the purpose for the transfusion, 
and so on. Rather than merely requiring 
medical students to memorize the Information 
In the knowledge base f already better organized 
and more relevant than much instruction), 
flexibility theory suggests that the information 
should be accessed from the context of various 
cases, because what was important in one case 
may be Irrelevant in another. 

Transfusion Hypertext 

The hypertext knowledge base that supports the 
outcomes described above is being constructed 
in Plus to run on the Macintosh family of 
computers possessing two or moie megabytes of 
RAM or under Windows 3.0 on 286 or 386 IBM 
machines with 2 or more megabytes of RAM. 
The software consists of five stacks of related 
cards, including: the knowledge base (known In 
the program as the textbook); a glossary of all 
terms that may be unfamiliar to the third year 
medical students for the whom the program is 
intended; required actions that the student 
may take including ordering screening tests, 
treating the patient, or getting more 
Information; a set of six primary, in-depth 
cases; and a set of 24 related cases. 

The program is oriented by the primary cases. 
That is, students access the program and are 
offered help in using the program. If the 
student is familiar with the operation of the 
program, then s/he is led to one of seven 
primary cases (see Figure 1 for case solution 
options). Access to any information is through 
these well defined cases, that is, case- 
dependent. The student must determine the 
information needed to take an action prior to 
taking that action. S/he has the option of 
accessing Information from a transfusion 
medicine textbook (a very common source of 
information in case resolution), iking 
q- .a. s of Important operatives In the case 
su^n a the attending physician, resident, 
patient, phlebotomist or blood bank director, 
ordering tests or comparing the current case to 
similar cases. The student may also criss-cross 
this case -based landscape by comparing the 
current one to similar cases. These 20 cases are 
le3s detailed than the primary cases. They 
have one or more more fields with common 
attributes. The more matching fields, the more 
similar are the cases. Each of these 
information sources provides a separate point 
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of view that perceives the case In a different 
way. The greater the level of similarity, the 
more useful the case should be to the medical 
student in helping him or her to take an 
appropriate action. These are the multiple 
perspectives that are normally available to a 
resident in solving a case. When the student 
takes an action, s/he is presented with 
feedback about the advisability of each action 
taken. 

Flexibility Theory Applied to Transfusion 
Medicine 

The tenets of flexibility theory described 
earlier are used to evaluate the transfusion 
medicine hypertext described above. 

Avoids oversimplifying instruction. 

Since the transfusion medicine knowledge base 
is ill-structured, like so much of medicine, no 
single event or case is prototypic. With nearly 
thirty different possible events and a large 
number of possible symptoms and 
pathophysiological responses to each event, 
the number of interactions is very large. The 
options available to the doctor, both in terms 
of information gathering and treatments 
further complicates the picture. This natural 
complexity needs to be conveyed by the 
instruction in order to avoid the reductive 
belief that transfusion medicine is a simple 
knowledge domain. 

Provides multiple representations of content. 

The information available to the medical 
student is varied. S/he may seek multiple 
perspectives from the patient, resident, 
attending physician, phlebotomlst, or even the 
blood bank director. S/he may consult similar 
cases that exhibit similar symptoms, events, or 
tendencies or a textbook which represents yet 
another perspective. S/he may take some 
action, such as ordering tests, prescribing 
treatment etc. The Information is represented 
in this hypertext on at least three different 
dimensions representing a large combination 
of representations. 

Emphasizes case-based instruction. 

Rather than basing instruction on a single 
example or case, it is Important that a variety 
of cases be used to illustrate the content 
domain. The cases In this transfusion 
medicine hypertext are very irregular, with 
different perspectives or factors assuming a 
more important role. There are six primary 
cases. More could be used, but curricular time 
constraints are imposed on the use of this 
material. !L l hese few primary cases Involve 
the studenu with multiple symptoms and 
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problems. Students will be able to compare 
these cases with 24 other similar but somewhat 
different cases. The knowledge, perspectives, 
and information are all case -driven, so that 
any Information is provided by the hypertext 
only in the context of a case. 

Context dependent knowledge. 

The importance of a clinical context for the 
acquisition of biomedical knowledge cannot be 
overstated. Biomedical knowledge is best 
acquired in relevant situations that are likely 
to be encountered by the student as a 
practitioner (Brown, Collins & Duguid, 1989). 
The cases provided In the transfusion medicine 
hypertext are based upon real experiences. The 
contexts are made more realistic by th"* 
availability of realistic advice from different 
people and realistic information from 
authoritative sources. 

Knowledge construction; not transmission. 

Learners must construct meaningful 
knowledge representations in order to adapt 
and use the information in novel situations. 
The transfusion medicine hypertext requires 
that users access and then associate 
Information in relevant, practical situations. 
In attempting to solve cases, learners build 
complex schemata that consists of procedural 
(how to) knowledge rather than an 
amalgamation of unrelated facts. Procedural 
knowledge is more readily transferable. 

Supports complexity. 

It is important that medical practitioners 
acquire non-compartmentalized knowledge 
based upon rigid, external knowledge 
structures. Users of the transfusion medicine 
hypertext are required to build their own 
knowledge structures in order to solve cases. 
The goal of the Instruction, as represented in 
the evaluation techniques, is identical to the 
instructional strategy -- solving relevant cases. 
In order to construct useful knowledge 
structures, learners need to compare and 
contrast the similarities and differences 
between cases. This is supported ty the 
multiple perspectives on each case and the 
related cases for comparison and contrast. 



Cone* «sion 

Physicians in training need to learn to 
diagnose illnesses and prescribe treatments. 
This involves clinically oriented problem 
solving. Rather than solving problems during 
instruction, medical students memorize 
information which is often poorly organized 
and is generally devoid of contextual relevance. 

7 2!;3 



This information is often not applied until a 
point much later in their training. The 
information that they are acquiring is full of 
complexity, irregularity, and inconsistency. 

This paper has presented a solution to many of 
these problems, cognitive flexibility theory, a 
solution that may have implications as a 
model for medical education. Cognitive 
flexibility theory \s able to represent the 
complexity of medical problems while 
permitting the learner to investigate the 
multiple perspectives represented in the 
knowledge domain in a problem solving 
context. 
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A NEW COMPUTER-BASED LEARNING EVNIRONMENT: 
THE HYPERMEDIA-ASSISTED KNOWLEDGE CONSTRUCTION SYSTEM 
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University of Houston, Clear Lake 



Abstract 

We are at transition point in history; between 
what has been traditionally known as the 
industrial age and what will probably become 
known as the age of technology. Some 
components our of society have successfully 
begun the transition between the two. One need 
only look at the integration of technology into 
the business arena and the design sciences. 
Spreadsheets and CAD systems have 
fundamentally changed the way these 
disciplines operate. The educational system 
has been notoriously slow in adapting to the 
needs of the technological world. Instructional 
systems of the future will be more adaptive 
largely because they will become more 
integrated, both in terms of the technology and 
user involvement. HAKCS, our vision of what 
CAI should be, is a prototypical learning 
environment under development at the 
University of Houston-Clear Lake which will, 
upon completion, elicit and store the mental 
models (semantic networks) of experts of 
various domains, permit other users to browse 
the components and organization of these 
mental models, and finally, actively engage the 
user in the construction of the mental model on 
screen and in their mind. HAKCS is said to be a 
hypermedia-assisted system, versus a 
traditional hypermedia system, since the 
primary goal of the system, knowledge 
construction, requires hypermedia features 
only indirectly. We see this system as the 
primitive but necessary first step toward a new 
generation of learning environments. 

Introduction 

Hypermedia systems (non-linear, associative 
multimedia) have received considerable 
attention as next generation learning and 
communication environments. Exponents 
have referred (metaphorically?) to such 
systems as the first "knowledge medium" 
(Stefik, 1986), "peripheral brains" (Carlson, 

1990) , a "psychological landscape" (Shirk, 

1991) , and "the next step in civilization" 
(Nelson, 1989). The comments do not, of 
course, refer to existing hypermedia systems, 
but rather to the more elaborate (and literarily 
safe) integrated systems of the future. This is as 
it should be, for there is currently little, if any, 



empirical support favoring hypermedia-based 
instructional or communication systems over 
traditional, linear systems or approaches 
(Jonassen, 1991; Locatis, Letourneau, & 
Banvard, 1989). An ode to Santayana would 
have us remember the familiar ring, and 
subsequent fate, of high-hype, low-empirical 
support technologies of days past-where is 
programmed instruction, anyway? 

Much of the excitement surrounding 
hypermedia arises amidst talk of a new 
medium of communication. Hypermedia is 
often represented as the latest in an 
evolutionary progression of communication 
media whose phylogeny Includes artifacts such 
as the spoken word, written word, still Images, 
and full-motion images (Dede & Palumbo, 
1991). With regard to education, however, we 
must be careful to evaluate hypermedia as a 
medium of education rather than a medium of 
communication-communication is, afterall, a 
necessary, but insufficient condition for 
learning. Gilbert (1982) wisely warns that 
overemphasis on delivery systems in training 
and education has underemphasized the 
development of pedagogy; i.e. new media have 
simply reinforced the transfer ^nd translation 
of bad pedagogy to new technological 
platforms. The (hyper) medium may influence 
the message, but it is not, by itself, going to 
teach us the message. 

DiSessa (1987) suggests that new insights into 
more effective pedagogy might come from 
developments in the cognitive sciences. As 
knowledge about knowledge increases, and as 
thinking becomes functionally reproducable 
on computer, opportunities for insights and 
research concerning the way we learn will be 
unprecedented. Technology-based 
instructional systems will presumably exploit 
this new information by actively 
incorporating, adapting, and perhaps even 
interfacing with user knowledge. It is possible 
that the now evident distinction between user 
and computer knowledge will in time blur- 
computers might become natural complements 
to our own cognitive shortcomings: short-term 
memory, recall inefficiencies, etc. The 
actualization of future knowledge -related 
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systems will require further development of at 
least three basic areas research within the 
cognitive sciences: (1) knowledge acquisition, 
(2) knowledge construction, and (3) human- 
computer Interaction (HC1). 

Knowledge acquisition refers to the elicltation 
of the functional structures of memories in 
long-term memory. This process, traditionally 
conducted by knowledge engineers for expert 
system development, is complicated by the 
obvious fact that the structures are 
unobservable. The knowledge engineer 
attempts to circumvent this little 
inconvenience by employing a variet}' of 
techniques such as associative recall, 
knowledge mapping, story telling, etc., but 
these techniques have proven time consuming, 
costly, and in the end, not very effective. 
Knowledge construction refers to the notion 
that learning involves the active construction 
and continual refinement of mental models. 
Generally, mental models are Internal models 
of reality comprised of something akin to 
semantic networks which, in turn, form 
structures which empower us to adapt to both 
novel and common situations (Mlnsky, 1986; 
Schank, 1990; Bartlett, 1932). Knowledge 
construction is receiving considerable 
empirical attention, albeit indirect, from a 
number of research domains in the cognitive 
sciences (mental models, memory, learning, 
etc.). Human-computer interaction, though 
probably not considered by many to fall within 
the cognitive sciences, will undoubtedly play a 
major role in the effective implementation of 
all technology-based systems. And, as the 
knowledge link between user and computer 
becomes more Intimate, HCI will exr nd in 
scope to accommodate the blur (arguably, it has 
already done this) and likely Join the plethora 
of disciplines composing cognitive science. 

The Hypermedla-AMltted Knowledge 
Construction System 

Semantic network theories (e.g. Collins & 
Quillian, 1972) suggest that the storage 
architecture of long-term memory is 
functionally equivalent to networks comprised 
of nodes (concepts) and pathways or links 
(directional, semantic associations between 
nodes). Hypermedia, based on a comparable 
node-link architecture, appears, at least 
superficially, to be an ideal medium for 
representing semantic network both 
functionally and visually. The trick is, of 
course, to achieve a relatively accurate transfer 
from long-term memory to the hyper network- 
then wonder what to do with it. We have 
wondered, and, with Jonassen, we believe the 



answer lies in the creation of a new kind of 
learning environment. 

HAKCS is a prototypical learning environment 
under development at the University of 
Houston-Clear Lake which will, upon 
completion, elicit and store the mental models 
(semantic networks) of experts of various 
domains, permit other users to browse the 
components and organization of these mental 
models, and finally, actively engage the user in 
the construction of the mental model on screen 
and in their mind. HAKCS is said to be a 
hypermedia-assisted system, versus a 
traditional hypermedia system, since the 
primary goal of the system, knowledge 
construction, requires hypermedia features 
only indirectly. We see this system as the 
primitive but necessary first step toward a new 
generation of learning environments. 

Knowledge Acquisition 

The knowledge acquisition component of 
HAKCS employs a top-down process of node 
development and association designation. The 
process begins with the primary domain node 
(the deepest level of abstraction within that 
particular semantic network) and proceeds to 
lesser degrees of abstraction. HAKCS queries 
the expert with each node addition to find out 
its relationships with other nodes in the 
network. The query parameters are limited to 
eight possible semantic relationships 
structured in a X {is related to) Y format. The 
semantic relationships supported are : 1. 
analogy (is similar to), 2. equivalence (is 
equivalent tc), 3. inclusion (is a part of), 4. 
definition (is defined as), 5. example (is an 
example of). 6. cause/effect (causes), 7. spatial 
relation (is spatially related to), and 8. 
temporal relation (is temporally related to). 
So, for example, if the system began with a 
primary domain node of quantum mechanics, 
HAKCS would ask for definitions, examples, 
analogies, and so on. until the expert was 
unable to provide further input. At this point, 
the system would begin the same process on the 
examples, analogies, etc.. Just entered. The 
process continues in this fashion until the 
network is completed, at which time the expert 
begins a process of iterative fine-tuning, adding 
nodes and/or relationships which were not 
recalled during the initial acquisition pass. 

The process is not particularly different from 
techniques now used by knowledge engineers. 
The basic approach is as follows: When person 
A attempts to explain X to Person B, person A 
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(1) Analogy {tt similar to) 

-functional 

■topographical 

-property 

-abstract 

•physical 

(2) Inclusion fas apart of ) 

-abstract 
-physical 

(3) Definition (is defined as) 

(4) Equivalence { » equivalent to ) 

•functional 

-topographical (5 senses) 
-property 

-abstract 

•physical 

(5) Example (is an example of) 

(6) Temporal (is temporally related to) 

-prior 

•concuntnt 

-post 

(7) Spatial (is spatially related to ) 

-above 
-below 
•beside 

-right 
•ten 

*- _* — 

-DCTOTC 

-behind 

(8) Cause/Effect (erases) 



Figure 1 Semantic Relationships 



might try to define X. If B still did not 
understand, A might provide an analogy. If B 
did not understand, A would continue to 
provide analogies, examples, spatial 
relationships, etc., until B can relate 
something to his or her mental model. Two 
Interesting things are happening here: person 
A was recalling a number of memories by 
semantic type, person B was provided these 
memories in a semantic context. Person A 
provided B with a glimpse of his mental model, 
and person B has made a connection between 
one of A's memories of one of B's own, 
providing B with some understanding of X. 
This process of communicating knowledge 
happens all of the time-it is, in fact, the 
essence of traditional Instruction. It should 
not be surprising, then, to find empirical 
evidence that students begin to assume the 
knowledge structures of their instructors 
(Shavelson, 1974). 

Although the acquisition process is greatly 
simplified by allowing only eight semantic 
associations, the intrinsic complexity of 
knowledge continues to make even this a fairly 
laborious endeavor. HAKCS can simplify the 
process somewhat by inferring a substantial 
number of the relationships automatically, 
reducing the front-end investment of the 
expert. For example, if the expert enters the 
following: 

Electron {is a part of) Atom 
Proton (is a part of) Atom 

HAKCS infers: 

Electron {is similar to {by abstract property}} 
Proton 

HAKCS would then create the appropriate 
analogical link. You may have noticed that the 
analogical inference was by abstract property. 
The eight types of semantic relationships are 
divisible into finer categories (see figure 1). The 
expert determines the grain of semantic detail 
necessary to convey what is in his or her 
semantic network. There will be occasions in 
which the expert will not feel comfortable 
operating within the confines of the semantic 
relationships provided, or perhaps wishes to 
qualify a node description, or whatever. 
HAKCS provides for these types of messages in 
what are called signposts. Signposts are simple 
textual and/or graphical additions which are 
connectable to nodes and links to make the 
user aware that the expert had something 
additional to share. As stated, this might be 
something as simple as a sketch or 
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qualification of a selection, or something like 
a personal anecdote or joke relevant to a 
particular concept. The idea is to impose 
sufficiently narrow semantic parameters to 
facilitate the acquisition, browsing, and 
construction of the mental model, but also 
create within those parameters a process 
sufficiently exhaustive to capture the essence of 
knowledge. 

Once the expert has completed the process of 
relating his or her mental model of a domain 
into the computer, system and Instructional 
designers work to translate many of the nodes 
into multimedia displays using computer 
animation, laser disc video, CD-ROM, etc. 
Upon completion, nodes might be as primitive 
as textual descriptions, or as sophisticated as 
full-fledged multimedia productions. In 
general, nodes are presented as real-time as 
possible, but can be augmented with process 
highlighters or similar devices to focus user 
attention on the particular phenomenon 
relevant to that node. 

The final step of the acquisition process is the 
recording of the expert's background, 
education, awards, current employment, and 
personal Interests. This information is made 
available to users in the form of a small 
biography, along with a digitized image of the 
expert, by simply clicking on the expert's name 
which has, at that point, become a permanent 
member of the expert log. 

Knowledge Browsing 

The HAKCS browser is designed to allow the 
user easy and intuitive access to, and 
exploration of, the expert mental model. 
Utilizing a combined onion-landscape 
metaphor, nodes at a particular level of 
abstraction are located on the surface of a 
sphere. Depth of abstraction is visually 
represented by the circumscription of spheres 
within spheres. i... .eepest level of 
abstraction, the sphere at the center of the 
onion, is the primary domain node. Nodes on 
the outermost layers will consist mostly of 
experiences (i.e. multimedia representation of 
experiences) called "primitives." 

Nodes are represented on an altered- 
perspectlve view of the landscape as three- 
dimensional cubes, the face of each displaying 
a picon corresponding to its contents. 
Semantic relationships shared by the nodes 
are represented as color-coded lines, a different 
color denoting different types of semantic 
relationships. Users browse this 
knowledgescape by pointing and clicking on 



one of the four directional arrows located in 
the center of the control panel (see figure 3). A 
window revealing the name of the node directly 
in front of the display is found just above the 
directional arrows. Unwanted complexity can 
be filtered by clicking on the appropriate 
buttons to the left of the control panel, or all 
relationships can be displayed at once. The 
buttons to the right of the control panel include 
a search feature which permits the user to 
quickly locate a specific node of Interest, a help 
feature providing on-line help, and a button to 
return to the launch platform (see figure 2). 

Circumscription of one sphere within another 
denotes an abstract, inclusive relationship 
(recursively Inclusive) between nodes on each 
of the two spheres-the nodes on the inner 
sphere being more abstract. Connections 
between the two spheres are visually 
represented as cones, similar in appearance to 
Robertson, Mackinlay, and Stuart's (1991) tree 
cones, forming on the outer sphere and coning 
Into a node on the inner sphere. The vertex of 
the cone, meeting at a node on the inner sphere 
(a composite node), expands upward so that the 
top of the cone encompasses all of the nodes 
and relationships of which the composite is 
comprised. For example, a composite node 
representing Doppler Effect might include 
nodes of passing trains and airplanes (laser 
disc video sequences), wavelength, frequency, 
etc. The visual representation of this 
relationship would have the vertex of the cone 
meeting the top of the node Doppler Effect, and 
expanding upward to encompass the nodes of 
passing trains and planes, wavelength, and so 
on (see figure 5). 

The tree cones are visually self-evident If users 
are looking at composite nodes, but If users are 
on a level on top of the tree cone, they would see 
what is called a semantic fence. The semantic 
fence is a structure resembling a small wall 
encircling all of ^ nodes comprising the 
composite node below. Upon coming across a 
semantic fence, users could choose to explore 
the deeper level of abstraction by clicking the 
down vertical arrow beneath the navigational 
arrows, and return by clicking the up vertical 
arrow. 
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The design rationale behind the 
knowledgescape browser has been the creation 
of a functional, intuitive, and interesting 
visual representation of a mental model. 
Unlike many hypermedia systems, the browser 
is not expected to substantially contribute to 
the learning process, but rather to provide a 
greater organizational context in which to view 
and think about the content of the model. 
While we think browsers might be useful in the 
creation of meta-mental models (mental 
models of mental models), it is believed that 
the mental models constructed from 
hypermedia environments will focus 
primarily on the navigational and contiguous 
aspects of the display, versus comprehension of 
the directional semantics of which the 
knowledge is constituted. We posit that the 
development of mental models centers around 
the visible interactions between components of 
systems. Mental models give us the ability to 
understand the syntax of what we perceive, but 
introspection of that model provides the 
semantics. Introspection, it is held, requires 
thought, probably consciously thought. Hence, 
transfer of the expert mental model residing in 
HAKCS' memory will require more than 
browsing, it will require thought about the 
knowledge displayed. It will, in effect, require 
the metacognltlon of the semantic structures of 
the representation. 

Knowledge Construction 

Introspection here refers to the contemplation 
of existing mental models. Craik (1943) argued 
that mental models operated in a manner 
functionally equivalent (or approaching 
functional equivalence) to processes being 
paralleled. Objective system processes, 
however, are without semantics until observed 
(Young, 1988). Hence, semantics requires more 
than the modelling of objective system 
processes. It is suggested that the creation of 
mental models deals primarily with the syntax 
or mechanics of reality, while meaning arises 
from the introspective comparisons between 
parts of, and/ or whole, mental models 
involving a different category of knowledge. 
This other category is best characterized as a 
priori associative knowledge, 
whereas the former is characterizable as a 
priori modelling knowledge. A full theoretical 
account is beyond the scope of this paper (see 
Li dwell. & Palumbo, In press), but a taste was 
necessary to understand the rationale of the 
knowledge construction component. 

The HAKCS knowledge construction set (see 
figure 4) has been designed to promote the 
metacognltlon of the semantic relationships 



on screen. Two nodes are displayed on screen 
as plcons with a number of buttons between 
them. The nodes, as in the browser, reveal 
their content when double-clicked. The 
buttons each represent a different semantic 
relationship. Users view the nodes in their 
various multimedia representations and 
attempt to determine the semantic 
relationships that they share. The user 
considers the nodes and the possible 
relationships and then selects the relationship 
of choice by clicking on the appropriate button. 
If the selection is correct, i.e. corresponds to the 
expert's selection, the system provides positive 
feedback, updates reference maps, and presents 
the user with new nodes. 

Reference maps are located below the the node 
plcons and display all nodes which have 
previously been linked correctly. Users can 
refer and review previously linked nodes of one 
node and then compare those with the 
reference map of the other node. It will often be 
the case that nodes will appear in both maps 
under different semantic categories. It is hoped 
that these maps will facilitate inferencing and, 
consequently, Improve selection performance. 
The extent to which the maps will prove useful, 
however, is questionable, and the maps are 
currently designed so that they can be revealed 
and concealed at the user's discretion 

If the user selects a relationship which does not 
correspond to the expert's, HAKCS Informs the 
user that the selection was incorrect, and then 
proceeds to change only one of the nodes on 
screen. The logic, as discussed previously, is 
that in attempting to communicate a novel 
concept to another person, one provides 
analogies, examples, and so on, until an 
association is made between the memory 
expressed and some memory In the person's 
mental model. HAKCS, too, provides nodes 
which are semantically related until a 
connection is made. Once an association is 
successfully established, HAKCS continues 
providing the network node by node until the 
user reconstructs the expert's mental model. In 
reconstructing the model on screen, it is 
believed that similar semantic structures will 
be constructed in the mind of the user (Lidwell 
& Palumbo, in press). 

The construction set currently supports two 
user modes: novice and expert. The novice 
mode provides neither all eight of the 
relationships, nor does it support the finer 
distinctions in each semantic category. The 
novice level is being designed primarily for use 
by children approximately twuve years of age 
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and older. The expert mode is being designed 
technical training *n Industry 2nd higher 
education. It supports all eight relationships 
and the finer distinction therein. In working 
with these semantic relationships explicitly, 
one might reasonably expect an Improvement 
in users' ability to associate experiences in 
non-HAKCS environments. If this is correct, 
HAKCS might do more than construct 
knowledge, it might make the construction 
process, in general, more efficient. 

Conclusion 

Philosopher Arthur Schopenhauer advocated 
what he called the "natural method" of 
education. To Schopenhauer, depth of 
knowledge required depth of experience. The 
whole of the experience, its essence, is only 
attainable through natural experiences (first- 
hand exposure). Natural education, then, was 
defined as an education comprised entirety of 
-atural experiences. The trend In education, 
even in Schopenhauer's time, was toward 
"artificial" experiences (second-hand 
experiences; e.g reading, listening about, etc.). 

The idea of a wholly natural education is 
obviously Impractical-or is it? Providing 
students with hands-on experience Is the 
current fad in education. Children are 
conducting actual scientific investigations, 
creating computer programs in a variety of 
programming languages, and building robots 
using construction kits like Lego/Logo. These 
activities are genuine, fulfilling experiences 
which get at the heart of Schopenhauer's 
philosophy. 

But what of those experiences that do not 
involve the physical construction or 
manipulation of something? What of 
experiences like seeing the grand canyon, 
hearing the fugues of Bach, or exploring the 
moons of Jupiter? How can such experiences be 
presented in an economically feasible and 
"natural" fashion? The answer, in a word, is 
hypermedia. Certainly Schopenhauer had no 
conception of the types of technologies that are 
or soon will be. Presently most experiences can 
be presented in near complete form through 
hypermedia technology. In the future, virtual 
realities will provide even more complete, and 
perhaps perfectly authentic experiences to the 
user. 

HAKCS circumvents the impracticalities of 
Schopenhauer's natural method by bringing 
the experiences to the user. The experiences are 
presented free from the chaos and distractions 
of truly natural experiences and permit system 



developers to emphasize the relevant elements. 
In this sense, technology has done nature one 
better. Presentations can be augmented 
through the use of process highlighters, voice 
guidance, and computer-enhanced video to 
concentrate user attention on those elements 
relevant to the linking task. 

Developments in technology and the cognitive 
sciences hold tremendous promise for training 
and education. Technology has proven a 
capable medium for transferring, processing, 
and recreating information. As our knowledge 
of the way we transfer, process, and recreate 
information increases, technology will 
effectively merge and become an interactive 
part of the process. HAKCS is seen as the first 
step in this direction. 
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Abstract 

Hypertext and Hypermedia have captured our 
imagination with their capabilities to support 
instruction, learning, and collaborative 
research. For IBM users taking the first step 
often means selecting a hypertext system that 
will provide the links between text, graphics, 
audio, and video. In this paper, emphasis is on 
a comparison of two hypertext systems based 
on the object-oriented paradigm. 

Introduction 

Enthusiasm for hypertext projects is growing 
rapidly due to the appearance of personal 
computer authoring environments and the 
Increased power of personal computers making 
them powerful enough to support hypertext. 
The fascination of being able to link 
documents and ideas, in a manner that 
heretofore could not be done, has reenergized 
researchers, teachers, and developers. Within 
their reach li possibilities for collaborative 
work with colleagues world-wide and the 
development of large multimedia databases. It 
is now possible to give students many diverse 
materials to explore that previously were not 
easily integrated into the traditional 
classroom, library, or museum setting 
(MacKnight, 1990). 

The task ahead is not a simple one. however. 
Gathering the appropriate texts, translations, 
dictionaries, video images, drawings, maps, 
other relevant documents, and selecting the 
appropriate tools illustrate the enormity of the 
task. The size of the information to be 
represented, how it is to be represented (text or 
a combination of text and graphics), whether 
the plans call for multi-or single-user system, 
and the characteristics and computer 
capabilities of potential users all impact on the 
hypertext system selected (Nielsen, 1990). (See 
also Halasz. 1988; Verreck and Lkoundi. 1990; 
andLocatis, 1991.) 

Knowing the limitations of the available 
hypermedia systems is essential in planning 
for the present and in being able to take 
advantage of emerging technologies in the 
future. The hypertext system chosen depends 
on what one is willing to sacrifice, since there 
is no universally best system (Nielsen. 1990). 



In this paper we describe some advantages of 
the object-oriented paradigm as implemented 
in Guide and Linkway -two hypertext systems 
that run on the IBM personal computer. 

Interface 

Guide and Linkway are interactive systems 
that give authors a new way to organize and 
present text, graphics, audio and video 
information. Both systems have graphic 
interfaces and several features in common: 

•functionality used to create objects, links, 
and applications, 

•editing objects and links, 

•managing objects, 

•configuring display features and attributes of 
the objects, and 
•navigation between objects. 

Features common to both systems that 
support the object oriented data model form the 
subject of our comparison. 

Description of Terms 

In Guide, a document is similar to Linkway's 
FOLDERS in that both form the basis of an 
application. A document contains' many 
frames, each of which can have several objects. 
Guide treats all elements-text, graphics, video 
and audio- -that can be selected individually or 
as a group as an object. The system also 
provides objects-reference, note, expansion, 
and command buttons— that link source and 
target information objects. Reference buttons 
create links to either another part of the 
document or to another document. Note 
buttons display context sensitive text or 
graphics information; command buttons 
execute other programs, open and close 
documents, and control the serial port; and 
expansion buttons function as menu items 
leading to subtopics and function as text and 
graphic frames. 

In a LinkWay application, FOLDERS contain 
several screens called PAGES. PAGES consist 
of different combinations of text, button, and 
graphic objects. Buttons perform such actions, 
such as creating links or text and graphic pop- 
up windows and executing external programs. 
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Menus and Customization 

Because of the simplicity of Its five item menu 
bar, the LinkWay menu system Is very intuitive 
to use (see Table 1). The FOLDER, PAGE, and 
OBJECT menus provide file, page, and object 
creation and management functions. The GOTO 
menus provides navigation functions and the 
OPTION menu offers a way to change font types, 
set backgrounds and foregrounds, execute DOS 
commands, and the like. 

Guide closely follows the MS-Windows menu 
structure and the Open Software Foundation 
Motif (OSF Motif) conventions for menu design 
(see Table 2). Due to the complexity of the 
functions provided and the requirements to 
follow the object selection and action 
paradigm, its menus are less intuitive for 
novices to use. Once users understand the new 
Interaction paradigm, however, the interface 
becomes much easier to use. All menu items 
have keyboard accelerators (short cuts), which 
are attractive to experts. There are no such 
shortcuts in LinkWay. 

The program window and document window 
display parameters are customizable in Guide- 
an Important feature when an application uses 
multiple windows. In contrast, Linkway's 
program window is fixed, filling the entire 
screen. 

Linkway has options for setting the 
background and foreground colors of a 
document. FOLDERS have a base page where 
elements-text, graphics or link objects- 
common to all PAGES reside and appear as a 
background in a document. Implementing this 
feature in Guide is complex and requires 
considerable knowledge of Guide and concepts 
of object-oriented interaction. The user has to 
create a document with special attributes to 
implement this feature. 

Object Creation and Editing 

In Guide, any text or graphic object in a 



document can be selected and made into a 
button. Selecting the destination object and 
using the Make link option In the Navigation 
menu establishes connections. Users enter text 
by clicking and typing in a frame. 

In Linkway's FOLDERS, the procedures for 
creating buttons and other objects appeal to the 
novice who is led to do the right thing. The user 
creates an object and fills in information-text, 
graphics, or destination-as prompted. 
Fromptlng, without a customizable option to 
turn the facility off, can annoy expert users. 

LinkWay has a simple paint program called 
LWPaint with limited drawing capabilities. In 
addition, pictures created with IBM 
Storyboard Plus 1 can be imported directly. 
Graphic objects Imported from other sources 
must be converted to the LWPaint format using 
LWCapture. All graphic objects are internal to 
the file. Guide lack internal graphics 
capability, but supports the importation of 
graphic elements through the MS-Windows 
Clipboard and has sizing and editing 
capability. 

To save space, a Guide graphic file can be made 
an external link, thereby, reducing document 
size and increasing performance. This Is a 
consideration if speed is an issue. To move or 
resize text and graphics, Guide uses the clipping 
frame attached to the object. Linkway requires 
a similar procedure but with more steps 
Involved. In Guide, commonly used text or 
graphic objects can be stored in a glossary and 
reused in any frame. Linkway does not have a 
similar feature. 

Text Search and Replace 

The FIND option, an unsophisticated feature in 
LinkWay, searches for a FOLDER name, field 
name, and search suing. In a successful 
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search, FIND displays the PAGE containing the 
object. In addition, the And option can be 
Incorporated as a button. Guide has a much 
more powerful Find option with parameters for 
case sensitivity, wild cards, and logical 
connectives. Furthermore, the user can specify 
documents, both linked and unlinked, and 
directories to search. A completed search 
displays a list of objects containing the 
specified search string. Then, the user can 
select an object and query its properties or go 
directly to the frame that contains the object. 
Although there is no replace feature in 
Linkway, the user can use FIND to identify the 
string, and make the correction by typing over 
the original. 

Navigation 

The ability to list a set of objects with their 
links is a helpful feature In Guide. In large 
documents, a list of links is useful In checking 
the existence or the Integrity of links and in 
assessing whether other objects maintain 
links to an object identified for deletion. 
(Sorely missed in Linkway is a comparable 
management feature.) A list of links can be 
used to create an index of all objects In the 
document. By clicking on any object in the 
index, one can reach the destination in a link. 
Since links can have names, Guide supports 
name links. 

Guide maintains a record of the last 100 objects 
the user has traversed. But, this dv «s not 
Include movement between frames. Therefore, 
skipping through frames without following 
links will not register in the backtrack record. 
Guide allows users to select four levels of 
backtracking, including references, 
replacements, and notes. LinkWay maintains 
a record of the last 10 pages traversed with the 
retrace option allowing backtracking. 

In Guide, users can specify behavioral 
attributes to an object. This feature is useful 
when navigating through the document, for it 
provides special effects, displays position, and 
documents position. LinkWay displays only 
the Page of the target object in a search. 

Summary 

Guide has more functionality than LinkWay 
(see Table 3), however, its interface is less 
Intuitive to people unfamiliar with the object 
action paradigm. 



LinkWay objects, but the functionality for 
importation and exportation is restricted to 
basic text and graphic objects. Linkway's 
interface is more suitable for beginners. 

Guide has features to list link names, create an 
index list, and has an extensive range of 
document display formatting commands. With 
respect to the latter, Linkway can set 
background or foreground colors only. 

Guide's search and replace capabilities are very 
powerful, while Linkway's are primitive. In 
addition, Guide has a freeze option to test the 
buttons. Nevertheless, LinkWay, has better 
facilities for switching between author and 
browse mode. Furthermore, LinkWay has a 
stand-alone browse mode. In contrast, Guide 
needs the Guide Reader and Windows to run a 
Guide application on another computer. 

Collaboration between authors is not possible, 
and access to their distributed databases is 
impractical due to data consistency issues, that 
is, there is no program to check that changes 
made by different authors doesn't destroy 
existing work. Both hypertext programs lack 
sophisticated features like exportation of 
mazes of links and nodes. The navigation map 
provided by Guide traces objects. To what 
extent this helps the user Is questionable. 
Confronting the user Is the need to visualize 
where in the document the specific objects to be 
altered are located. 

Guide and LinkWay give us inexpensive ways to 
create hypertext on-line help, on-line manuals, 
and to create instructional Interfaces to 
laserdiscs. For projects involving large 
amounts of text, LinkWay offers the potential 
for accessing CD-ROM. However, people who 
are unfamiliar with hypermedia concepts can 
benefit from starting with LinkWay and then 
migrating to Guide or remaining with Linkway 
and added M-Motion to increase the 
functionality of LinkWay. 



Guide permits exportation and importation of 
objects, editing of objects, and, like LinkWay, 
printing the entire document. Users can edit 
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Abstract 

The literature implies a trend towards an 
increased use of interactive multimedia 
technologies for Instruction. The increased 
availability of moderate cost, good quality, 
digital audio computer cards and computers 
with built-in audio capability has enabled 
trainers and educators to realize the potential 
of random access audio for multimedia 
applications. 

In this paper, the author discusses the use of 
digital audio in computer programs, outlines 
considerations for recording digital audio, and 
offers design guidelines for Implementation of 
audio in multimedia instruction. 

Introduction 

Until recently, most computer-based training 
(CBT) programs concentrated on the visual 
presentation mode to the exclusion of the 
auditory mode because of hardware 
constraints. In fact, many textbook authors 
that purport to teach the design and 
development of CBT fail to even mention the 
auditory channel and certainly do not focus on 
its potential. 

The advent of the Macintosh computer with 
audio capability and the Increased availability 
of moderate cost, good quality digital audio 
cards for IBM-PC compatible computers have 
enabled trainers and educators to realize the 
potential of random access audio for CBT and 
other multimedia applications. Few 
guidelines, however, exist for the 
Implementation of digital audio with computer 
instruction. 

Overview of Digital Audio 

At its Inception in the 1960s, audio in CBT was 
utilized primarily as a monotonous feature 
(such as a beep or a buzz) for Inappropriate 
input by the computer user. As software and 
hardware improved, audio in the form of music 
was added for reinforcement of correct answers 
as well as to fill the dead space created by slow 
access times to various program options. 

With the advancement of interactive videodisc 
(IVD) technology, widespread use of audio 
became feasible. In this format, audio was used 
to provide spoken Instructions, include sound 



effects, create scenarios, explain concepts, and 
prescribe remediation (Pratt & Trainor, 1990). 
However, audio on laser videodiscs has some 
constraints. Videodisc production Is expensive 
and audio is "tied" to a particular video 
segment. In addition, once mastered, the video 
and audio cannot be changed or relocated on 
the disc. 

The introduction of digital audio in 1982 
created circumstances necessary for effective 
integration of synchronized, random audio 
Into multimedia instruction. Modern digital 
audio computer cards enable multimedia 
developers to record, store, edit, and play back 
segments of audio. "Access to the segments is 
rapid (on the order of milliseconds), and the 
amount of information controlled is large (on 
the order of hours)" (Davis, 1989, p. 1). Control 
of these segments through computer software 
programs is precise and reliable. 

In order for a computer to have the capability 
to produce speech, hardware capable of 
producing and amplifying necessary sounds 
must be present. For some computers (like the 
new Macintosh), hardware is built into the 
computer; for other computers, an add-on 
board (card) and peripheral voice box are 
required. In most cases, speech cards are not 
interchangeable; therefore, different speech 
cards might be needed to run various speech 
programs on the same computer. These 
hardware considerations definitely have had a 
negative Impact on the use of audio. 

Currently, there are three methods employed to 
produce computerized speech: (1) text-to- 
speech synthesis, (2) linear predictive coding, 
and (3) digitized sound. Methods vary 
tremendously in quality, cost, applications, 
and hardware requirements. 

The most memory- efficient approach to 
computerized speech is text-to-speech 
synthesis. In this approach, language is 
defined as a fixed set of sounds, and computer 
algorithms are used to "pronounce" printed text 
into spoken output without human Interaction 
of any kind (Greene, Logan & Pisonl, 1986). 
Because text-to-speech synthesis works with a 
set number of sounds and algorithms, it does 
not require a great deal of computer memory. 
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Text-to-speech sound Is ideal for "talking" word 
processors that can "read" anything that is 
typed in. Most speech cards for Apple II 
computers provide text-to- speech capabilities, 
and many software applications for this 
technology are targeted toward small children. 
The disadvantage of the text-to-speech 
synthesis method is that it is very unnatural 
and mechanical sounding. This is a particular 
problem in educational settings where realistic 
speech "could aid considerably in the language 
development process" (Salpeter, 1988, p. 32). 

A second approach to computer speech is 
Linear Predictive Coding (LPC). This method 
begins with a human voice recording that is 
then digitized and compressed. A special 
speech synthesis chip is required to reconstruct 
words. The memory required by LPC speech is 
minimal compared to less compressed digitized 
sound and the quality is not as realistic; 
however, LPC speech is considerably easier to 
understand than words produced by a text-to- 
speech synthesizer. Many audio cards 
(especially for Apple II computers) offer LPC 
capabilities, but applications are limited to 
programs using a controlled vocabulary. 

The best quality of computerized speech is 
produced by digitizing with a technique similar 
to that used for creating digital audio 
recordings on compact discs. This technology 
is extremely realistic and natural sounding, 
and its educational potential is enormous. 
Digitized sounds are recorded (with the use of a 
digital audio card) and stored as flies on either 
a computer disk or compact disc. Files can 
then be controlled by a computer program, 
retrieved, and played instantly. 

There are, however, major roadblocks to the 
integration of digital audio. The primary 
problem is that the process of storing digitized 
sounds requires an enormous amount of disk 
space. In addition, developers must now be 
concerned with both the dynamic range and 
frequency of sounds and their resultant effects 
on the computer memory requirements. 



Frequency and Dynamic Ranges 

The human ear can detect sounds ranging in 
frequency of 25 Hertz to over 16,000 Hertz (16 
KHz). In addition, the ear can handle volumes 
(dynamic ranges) from one decibel (1 db) to 120 
decibels; but duplicating huge variations of 
frequencies and volumes is a problem for a 
digital audio system. Davis (1989) outlined 
frequency and dynamic ranges provided by 
various systems in Table 1. 

Because files generated by audio are 
proportional to the frequency range and 
dynamic range of the recording, these factors 
must be constrained to keep files to a 
manageable size. Most audio cards, therefore, 
limit the dynamic range to about 40 dB. When 
recording audio files, care must be taken to 
keep the amplitude of the digitized signal 
within the range of the audio card or distortion 
will occur. 

The limitation of the frequency range is not as 
critical as the dynamic range. Even though the 
human ear can detect sounds ranging from 25 
Hz to 16 KHz, most sounds necessary for a 
multimedia program fall within a much 
smaller range. For example, the majority of 
audio energy In human speech falls within the 
range of 200 Hz to 2500 Hz (Davis, 1989). 
Therefore, transmissions that focus on the 
human voice, such as telephone, AM radio, and 
multimedia programs, can be limited to 4 KHz 
(see Table 1). 

Another challenge encountered when digitizing 
sound is that the rate at which a signal is 
sampled (sampling rate) must be twice the 
highest frequency required; if a 4 KHz 
frequency is desired, the audio must be 
recorded at 8 KHz. Selecting an optimal 
sampling rate usually Involves a trade-off (the 
higher the sampling rate, the more natural the 
speech, but the more computer storage space 



AUDIO SYSTEM 


DYNAMIC RANGE 


FREQUENCY RANGE 


Telephone 
AM Radio 
FM Radio 
Good records 
Compact Discs 
Digital Audio (PC) 
Human ear 


35 dB 
45 dB 
60 dB 
70 dB 
95 dB 
(variable) 
120 dB 


100Hzto4KHz 
40 Hz to 4 KHz 
20 Hz to 16 KHz 
20 Hz to 22 KHz 
20 Hz to 22 KHz 
20 Hz to (variable) 
25 Hz to 16 KHz 



TABLE 1 DYNAMIC RANGES AND FT<EQUENCY RANGES 
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SAMPLING RATE 


STORAGE FOR 1 
SECOND OF SOUND 


SECONDS OF SOUND 
PER 1 MEGABYTE 


22 KHz 


22 Kbytes 


45 seconds 


11 KHz 


11 Kbytes 


90 seconds 


7 KHz 


7 Kbytes 


135 seconds 


5 KHz 


5 Kbytes 


180 seconds 



TABLE 2 SAMPLING RATE AND STORAGE 



COMPANY 


RATE 


CONTACT 


DATE 


Regency, Inc. 


7-8 KHz. 


Richard Davis 


8/30/90 


Interactive Mult. Indus. Assoc. 


7-8 KHz. 


David 


8/24/90 


MetaMedia 




McFarllng 




8 KHz. 


Robert Hamlin 


8/24/90 


Institute of Simulation & 


12 KHz. 


Peter Klncald 


9/24/90 


Training 






WICAT 


12 KHz. 


James Olson 


9/03/90 


Analysis & Technology 


12 KHz. 


Hugh Fisher 


9/15/90 



TABLE 3 RECOMMENDED SAMPLING RATES FOR AUDIO 



required). A typical one-hour multimedia 
course can require many megabytes of storage 
for audio. Table 2 represents the trade-off 
involved in the sampling rate/storage 
(Schmidt 1989): 

Results of an informal survey conducted to 
determine sampling rates being used for 
multimedia revealed that the majority of 
development is being recorded between 7 and 12 
KHz. Table 3 presents sampling rates 
recommended by companies contacted. 

Recording Audio 

Most audio used for multimedia is recorded 
with an IBM (or compatible) or a Macintosh 
computer. In the IBM realm, many different 
audio record/playback cards are available at a 
reasonable price (such as the Sound Blaster 
card). Most cards offer a limited range of 
different sampling rates (such as 8, 12 or 16 
KHz). It is Important to ensure that a card used 
to record sounds is compatible with the card 
used for playback. 

For several years, Macintosh computers 
contained a built-in digital-to-analog 
converter to play recorded music and speech; 
however, In order to record sounds, a 
peripheral, such as a MacRecorderby Farallon, 
was required. The latest Macintosh computers 
(Macintosh LC and II si) offer both recording 
and playback ability. 

The process of recording digital audio is very 
similar to recording with a cassette recorder - 



except that the controls on are the computer 
monitor rather than the tape recorder. Most 
audio cards allow input from a line source (a 
pre-amp, tape-recorder, or microphone 
amplifier). An RCA phono connector is 
generally included, providing interconnection 
with common audio equipment. There are two 
methods for recording audio-tape transfer or 
direct record. 

In the tape transfer technique, audio is 
recorded first onto a tape and then transfered 
to the cor??uter. The advantage of this method 
is that the recording and editing can take place 
in a quite room - away from noise generated by 
the disk drive, computer fan, etc. The taped 
information is then transfered directly to the 
computer from the tape recorder. 

Another alternative is to record the audio 
directly to the computer by using a microphone 
or microphone amplifier attached to the 
computer. This method Is quicker and 
eliminates the need to use a tape recorder; 
however, it is less flexible and more prone to 
excess noise. 

After the audio is recorded, it is easy to "play" it 
with the software program provided with the 
audio card or peripheral. To replace the file, 
simply record again using the same file name. 
It is also relatively easy to access digital audio 
from authoring systems, such as HyperCard, 
LinkWay or TenCORE. If an authoring program 
does not Include drivers for a particular audio 
card, it can generally be accessed through DOS. 



Guidelines for Using Digital Audio in 
, Multimedia Instruction 

Based on experience with digital audio in both 
the IBM and Macintosh worlds, the following 
guidelines are offered: 

•Balance the quality of audio sources. It is 
generally best not to mix high quality analog 
recordings with low quality digital recordings 
in the same lesson. 

•Separate spoken phrases with a short silence 
(about 1/2 second). Adding a short silence 
helps to make sequences sound less choppy 
and unnatural. 

•Develop on baseline hardware. Digital audio 
files are loaded into RAM before they are 
played. If your delivery hardware has less 
RAM than the development hardware, you 
may lose some of the audio. 

•Use a professional narrator. Professional 
voices can help to control the recording level 
and can add interest and emphasis to 
narration. In most cases, the cost is well 
worth the added expense. 

•Keep narration short. Approximately seven 
seconds is recommended as a maximum for a 
single piece of narration. Chunking is Just as 
Important for presenting audio information 
as it is for textual information. 

•Include visual cues. Some learners are very 
uncomfortable with strictly audio 
information. 

•Avoid synchronization. Because of the 
variable time that it takes to load audio files 
Into RAM and play them, it is very difficult to 
synchronize effectively with digital files. 

•Include a "repeat audio" option. In most 
authoring systems, it is easy to include an 
option that will repeat the audio segment for 
the learner, without having to branch back to 
the previous screen. 

•Include an "interrupt audio" option. An option 
that allows the user to proceed without 
listening to the entire audio sequence Is also 
desirable. This feature is especially Important 
If the lesson Is used for review purposes. 

Conclusion 

The hardware and software tools necessary to 
integrate digital audio into multimedia 
applications are now readily available. 
Preliminary studies in the field indicate that 
audio in digit"! lormat can be an effective tool 
for enhancing instruction (Barron. 1991; 
Torgensen, Waters. Cohen & Torgesen. 1988; 
Weiner, 1991). Instructional designers and 
developers can incorporate this technology to 
deliver the best possible multimedia 
instruction. 
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HIGHLY INTERACTIVE PROGRAMS WITH VIDEO FOR LEARNING 

LANGUAGES 
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ABSTRACT 

This paper discusses strategies for achieving 
high quality interactions in multimedia 
materials, especially those for language 
learning with video. We also describe our 
design process for highly interactive learning 
material. The major emphasis is on 
pedagogical approaches, rather than 
technical details. 

Our Understanding Spoken Japanese Project 
at the University of California, Irvine, is an 
example of a system developed with the 
strategies given in this paper. The support 
for this project comes from the Nippon 
Television Network Cultural Society, Tokyo. 
Japan. 

The techniques in this paper are usable in 
all language learning. We give examples from 
the learning of English as a second language. 

INTRODUCTION 

This paper describes strategies for highly 
interactive software for helping students to 
comprehend spoken languages, using video 
excerpts chosen for language features. This 
paper is based on an earlier paper describing 
our current project. 

Understanding Spoken Japanese |Yoshii, 
19911. The ideas in this paper, though, apply 
to learning any language, not Just Japanese. 

In the systems we envision, both computers 
and video are involved. The video is 
completely under the control of the program. 
The typical hardware is a personal 
computer, with either a laserdisc or a CD- 
ROM; some situations may use both. But this 
paper will have little concern with such 
technical details. Our interest is in the 
pedagogical Issues. 

The key phrase is "highly interactive." The 
student does NOT spend extended periods 
reading text, looking at video, or otherwise 
not participating actively in the learning 
experience. Further, the student does not 
move at random through the program. The 



program is aware of the student's progress, 
and adapts to each individual's capabilities, 
guiding students through learning materials 
as appropriate to their individual levels. 

Later sections give fuller details on the term 
"highly interactive." We do not regard most 
interactive video as highly interactive, 
including material for learning languages. 

COMPUTERS AND LEARNING LANGUAGES 

The use of interactive technology in learning 
languages is one of the oldest uses of the 
computer in instructional situations. The 
German course developed by IBM in the 
1960s was an early example of interactive 
learning material. Soon after this Patrick 
Suppes at Stanford University developed 
several full computer-based courses In 
languages, Including a course in Armenian. 

Although these early attempts were at the 
full-course level, recent attempts have been 
only fragments of materials for traditional 
courses. While funding has existed for 
developing non-interactive courses, such as 
video courses (French in Action [Capretz, 
1987], for example), no funding has gone to 
developing full, highly Interactive courses, as 
far as we know. One might say that we have 
moved backwards In our scale of coverage. 

The early full language courses did not use 
video, because the laser recording 
technologies were not available. But since 
the development of laserdisc and CD-ROM 
players the technology has been available. 
The value of sound and video for learning a 
language has long been recognized, well 
before the use of computers or laserdiscs. So 
it is not surprising that new possibilities are 
being explored now — although not, as far 
as we know, at the full course level. This 
paper does not attempt to review these 
developments; it focuses on pedagogical 
strategies that may be useful in such 
material. (For a list of computer-aided 
language learning materials and videodiscs, 
see [Vance, 19861 and [Rubin, 1990].) 
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Unfortunately much work today uses what 
we consider weak Interaction. The next 
section considers weak and strong 
Interaction types. 

HIGHLY INTERACTIVE LEARNING 
MATERIAL 

The term "interactive" Is often used in 
describing technologically advanced learning 
units. But much of this material Is only 
minimally interactive, as measured by the 
criteria we describe. 

Interaction is not a "yes" or "no" question. 
We must discuss the degree and the quality of 
the interaction. One of us has discussed the 
interaction spectrum In other papers [Bork, 
1979, Bork, 19871. 

First we describe strategies that limit the 
quality of Interaction. Then we describe 
strategies which limit the frequency of 
interaction. They are poor pedagogical 
choices in learning or training. 

Unfortunately these poor strategies are often 
used with Interactive language learning 
video. There could be at least two reasons 
for this: 

• Since video can only display, it is 
Inherently non-interactive. Linear video 
developers, given even the slightest bit of 
Interaction, discover a facility they never 
had before. Hence they become excited, 
even though the degree of Interaction is 
slight compared to what is possible. Often 
they have no idea of the full possibilities. 

•Being accustomed to lecturing, to multiple 
choice and textbooks, teachers continue to 
design courses that use these formats. 
Although the latest developments in 
language teaching [Krashen, 1983] 
emphasize frequent conversational 
Interactions with the students, classroom 
techniques have not yet reached interactive 
video courses. 

We will conclude this section with strategies 
leading to interaction high in both frequency 
and quality. Some of these features are 
general, but some refer directly to learning 
languages. 

ONE-SIDED INTERACTIONS 

The strategies listed here could lead to 
frequent interactions in learning material; 
but the very limited amount of the 
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information about student learning, and its 
low quality, prevent them from being called 
"highly interactive" strategies. 

Multiple Choice 

Multiple choice is a major example of low 
quality Interaction. Multiple choice does not 
correspond to the real world: In most 
situations many possibilities are available. 
A mechanic about to repair your car's 
transmission has more than four possible 
choices. 

Multiple choice is a poor way of determining 
whether anyone has a capability or not, 
except In the rare situations that have few 
possibilities to begin with. Because of the 
broad variety of features In languages, 
numerous errors and misconceptions, and 
varying degrees of comprehension, can be 
expected in answers to any non-trivial 
question. Adequate analysis of language 
comprehension is beyond the scope of 
multiple choice. As its popular student 
name, "multiple guess," suggests, students 
often guess randomly, hardly trying to solve 
the problem. Self-reliance is essential if 
language comprehension skills are to be 
acquired. 

Despite these failings, many systems 
continue to use multiple choice because of 
assumed difficulties of comprehensive 
analysis of student response. We, in our 
work, find such analysis to be a tractable 
problem. 

Student-Chosen Paths 

Some learning systems present the student 
with the list of items to be explored, but they 
make little attempt to direct the student, and 
no attempt to find out if the student is 
learning anything, or what learning 
problems the student has. Lacking this 
information, the system can offer no 
Individualized help, leaving the student to 
choose some path through the material, 
without direction. A critical aspect of 
learning is determining how the student is 
progressing, and using that information to 
make subsequent pedagogical choices. 
Computer-based learning material provides 
wonderful opportunities for this. But they 
are seldom realized. 

This strategy is often Implemented using 
hypermedia tools. Examples are Brown 
University's hypermedia courses, and those 
described In (Underwood, 19891. Each screen 
may have many words tha' can be triggered 
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for more information, or may have 
navigation buttons and icons leading to 
other parts of the 
program. 

For example, in a language-learning 
program, there may be parts which give the 
video in whole or in part; translations (or 
transcriptions) of lines in the video; reviews 
of the lesson; quizzes; and on-line 
dictionaries. The student decides which parts 
to visit, and when. The program, knowing 
noliiing of the student's capabilities, is 
allowing random choices. 

Although there is little empirical 
information, we believe the confused history 
of a student's passage through anunguided 
environment makes it difficult to assess the 
student's progress and problems, as needed to 
determine and provide the necessary help. 
The mixture of the two strategies is needed to 
achieve guided exploration. We hold that 
both freedom and discipline are needed in 
learning environments, and that without 
both, the student will not achieve the best 
learning. 

INFREQUENT INTERACTIONS 

The strategies listed here limit the frequency 
of interactions, leading to a passive learning 
situation. As a result, even with some high- 
quality interactions, the amount of 
information gathered from the student is 
limited. 

Passive Video 

Video-based language courses often show 
long video segments to the students before 
they have any interaction with the medium 
or a human teacher. This tends to exclude 
students from the activity, since they cannot 
concentrate on extended presentations in an 
unfamiliar language. We found that many 
students who helped evaluate our 
Understanding Spoken Japanese system 
disliked presentation of videos containing 
more than several sentences, where they 
remained passive and unlnvolved. Too much 
information in the video leaves the student 
uncertain on which of it he or she is 
supposed to concentrate. 

Some interactive video systems choose first 
to present the video passively, followed by 
interactions and remedial scenes. But we 
prefer to bring students to comprehend the 
whole story, if they so desire, through a 
sequence of interactions dealing with 
appropriate excerpts. Naturally, different 



students gain better comprehension from 
differently chosen excerpts, of differing 
lengths and in differing orders. 

Lecturing 

Although lecturing is a form of presentation 
with no Individualization and little or no 
Interaction, it is nevertheless often seen in 
many technology-based learning systems. 

Lecturing often takes the form of long 
textual explanations, perhaps entire 
paragraphs of text, intended to be read 
passively. It also tends to include 
vocabulary lists; grammar rules; and 
translations (and transcriptions) of speech 
from the video. 

Computer-based materials should maximize 
the advantage of personal, individualized 
sessions, as commonly used by private 
tutors. We should take full advantage of 
their individual nature, not relying on 
methods created to handle laige audiences. 

HIGHLY INTERACTIVE STRATEGIES 

In contrast to the strategies already listed, 
the ones now described keep material highly 
interactive. These allow the system to collect 
information about the student's progress, and 
use it to assist the student. Also, the 
frequent use of these strategies in the 
material keep the student engaged and 
concentrating. 

Overall Organization 

One way to achieve a high quality of 
interactivity is to organize the learning 
material as a carefully designed series of 
exercises of different types, centered on short 
video excerpts that demonstrate language 
objectives. In each exercise each student 
response should be analyzed, and 
individualized help sequences provided 
which make use of additional material 
(speech, pictures, etc.) beyond the video itself. 

The major components of this organization 
are: 

•Guided Exploration 

The general strategy for moving 
through the series of exercises is for 
the program to decide presentation 
order, based on its analysis of the 
student's progress. In certain places, 
the student may be allowed to decide: 

- the order for independent 
exercises; 

- whether to retry a prior, reli :ed 
exercise; 
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- whether to repeat the video or 
audio that was played as the basis 
of the current question. 
Interaction and Video Excerpt* 
Many exercises may start with a 
short video (visual, audio, or both), 
demonstrating a learning objective; 
the video should typically be under a 
minute. Subsequent interactions will 
be based on it. 

Even when the whole video is made 
available, the student should have 
some control over the video, rather 
than being obliged to wait until it has 
played out. In material working on 
spoken language comprehension, using 
short excerpts for the video 
presentation reduces any need for 
subtitles and vocabulary lists. 
Subtitles can be useful where the 
student shows great comprehension 
difficulties. This lets the student 
concentrate on listening and 
comprehending, rather than on 
reading. 

Individualized Help, Not Lecturing 

The series of exercises should be 
designed to lead the student to 
discovering language elements without 
long explanations. A student having 
continual difficulty with some point 
should receive increasing levels of 
help. 

When the student cannot get through 
an exercise successfully, a short, 
Immediately relevant explanation 
should be offered, supplemented by 
additional audio and visual material. 
Then further questions should check 
the student's understanding. 

The text on the screen, In either 
language, should be brief, no more 
than a few lines. It should be 
presented at a speed that lets the 
student read it as it appears. The 
sparse use of text helps to avoid 
lectures on the screen, and it lets the 
student read the text thoroughly, 
rather than starting to scan it, as 
tends to happen with, for example, 
newspapers. 
Different Levels for Different Students 
A program should analyze the 
student's progress and problems based 
on the numerous Interactions with 
the student, and determine what level 
Is appropriate. The program should 



supply various tracks or paths of 
differing complexities, and both Judge 
on which path the student should 
start, and whether, at various points 
along each track, another track would 
be more appropriate. 

We want the program to respond 
appropriately to a wide variety of 
student answers. Within any exercise, 
we should allow a great variety of 
answers, analyzing a great breadth of 
difficulties. 

Types of Exercises 

There should be a variety of ways for the 
student to interact, to motivate the student 
further, and to accomodate different types of 
information needed about the student's 
progress. Furthermore, different aspects of 
the language being learned require different 
types of exercise. 

For each type of exercise to be described 
below, we give an example from learning 
English as a second language. We will focus 
on the English articles, which are stated to 
be the most difficult feature in learning 
English [Covltt, 1976]. 

• FREE-FORM ENTRY: 

The approach most similar to a 
conversation between a tutor and the 
student is a textual answer in 
response to a question, In either 
English or the language being learned. 

No format should be imposed on the 
entry. Reasonable misspellings, 
including phonetic spellings, for each 
exercise should be determined and 
accepted. 

Questions could explore different 
aspects of language learning, such as 
listening comprehension, sentence 
construction, and vocabulary. Subject 
matter may come 
from either: 

- the video Just seen; 

- examples beyond the video. 

FOR EXAMPLE: 

The program shows a video in which 
a girl's cat gets lost. The girl goes 
looking for it, and she asks "Have 
you seen a cat?'. 

- The program asks the student to 
reproduce the girl's question as she 
phrased it. 
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- Anticipated answers Include: the 
completely correct answer; 
paraphrases of It; ungrammatical 
renderings of It; mishearings and 
misspellings of the words; and 
different meanings, possibly 
arising from Incorrect article 
usage. 

- For Incorrect articles used In the 
response, the program asks the 
student to pay attention to the 
article, possibly by replaying the 
video, or by playing additional 
audio that emphasizes the noun 
phrase. The program gives hints 
by referring to the context In 
which the girl chose the article. It 
also confirms the correct answer 
by describing why the particular 
article should or should not be 
used, again referring to the 
context. 

•ROLE FLAYING 

Free-form entry may also be used in 
role-playing exercises. Here, rather 
than answering questions, the student 
enters one side of a conversation, or 
even initiates It under the lesson's 
guidance. The conversation would 
usually be with some character 
simulated by the lesson, usually on 
the basis of characters already seen in 
the video. Questions can either by 
written or spoken. At present, 
answers can only be written. 

Playing a role gives the student an 
opportunity to practice conversations. 
This cannot be achieved with such 
simplistic strategies as multiple 
choice questions. 

FOR EXAMPLE: 

The program sets up a small story 
visually. The simulated character 
comes into a room and asks questions 
about another family member (It Is 
assumed that the character and the 
student live in the same house). The 
student must respond to those 
questions, using the correct English 
articles for the context. 

- ''Where did Mary go?": a correct 
answer would be 'To THE beach." 

- 'Which towel did she take with 
her?": a correct answer would be 
"THE red one." 

We look forward to technology letting 



students speak, with the program 
having full analytic capabilities. This 
will further increase interactivity. 



•ICONS and BUTTONS 

Buttons are useful in letting the 
student choose objects on the screen. 

FOR EXAMPLE: 

The student clicks on Items described 
by a particular use of articles In a 
particular context. This would let the 
student show understanding of 
Individual articles by taking the 
appropriate actions. Example 
Instructions to the student are: 

- "SHOW ME THE APPLES", 

- "SHOW ME AN APPLE", 

- 'INDICATE THE WHITE HOUSE IN 
WASHINGTON", 

- "POINT TO A WHITE HOUSE." 

Conversely, one can supply word 
buttons so the student can construct 
sentences to cause an action when the 
sentence makes sense. 

FOR EXAMPLE: 

With many objects shown on the 
screen, the student presses buttons to 
construct such sentences as "Show me 
the green house on the pond." The 
program responds to meaningful 
sentences by highlighting the specified 
object or objects - In this case, the 
single green house on the pond. The 
program responds to meaningless 
sentences by asking for clarification. 
For example, if several green houses 
were on the pond when the student 
created this sentence, the program 
would ask which of them was meant. 

Buttons limit the number of "trlviar 
errors or paraphrases the student can 
make with component words, leading 
to more reliable analysis of answers. 
This approach is similar to 
"scrambled sentence" exercises, but 
the student is able to get Immediate 
results by seeing the action described 
by the student's sentence, or hearing 
the sentence. The choice of actions, 
requiring the student's understanding, 
has Immediate visible consequences 
in a nearly physical world. 
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•VIDEO EXERCISES 

The student can be asked to 
manipulate the video. For example, 
students can be asked to Identify a 
specified word or phrase spoken in 
the video. This can be done by 
pressing a key or a button as the 
phvase is heard. 

The student can recognize a word or 
phrase within the normal stream of 
sound, where the student may not be 
expecting It, rather than as an 
Isolated example spoken specifically 
for student Identification. 

Varieties of this exercise Include: 

- finding all occurrences of a 
specified word or phrase; 

- finding all such occurrences that 
have a common meaning, where 
other occurrences may have 
different meanings 

- finding variations on a specified 
phrase. 

FOR EXAMPLE: 

To help the student pay closer 
attention to articles as they are 
spoken, play a video, and ask the 
student to press the space bar every 
time an English article is heard. At 
each point, ask the student to identify 
the article, and explain why it is used 
in that context. 

We expect imaginative design by 
excellent language teachers to produce 
more varieties. 

Usually the video would be started for 
the student. It may be restarted If an 
adequate number of occurrences have 
not yet been identified. 

One variation is to give the student 
buttons to move the video to any 
desired position, where it will resume 
play; so the student controls the 
finding of the phrase. This method 
can be used not only for audio, but 
for visual information on culture 
related to the language. 

Finally, a component of each lesson 
can be free-play, with full video 
control given to the student. The 
student may replay all parts of the 
video as often as desired. This gives 
an opportunity to place what they 



have Just acquired back in Its 
context, and to exercise their hearing 
as much as they feel they need. 

PEDAGOGICAL DESIGN PROCESS 

How does one generate highly interactive 
material for learning languages, based on the 
strategies just discussed? The key part of the 
project Is the PEDAGOGICAL DESIGN of the 
learning material, by skilled teachers of the 
language being taught. 

The following points are the major aspects 
to consider in design: 

The selection of the best possible teachers 
is critical. 

• The teachers selected should be 
Involved, on a daily basis, with the 
students to whom the units are 
directed. 

•The teachers should LIKE students. 

• The teachers should like interactive 
approaches to learning. They should 
work frequently individually with 
student. 

•The teachers should like conversational 
and communicative approaches to 
language learning. 

• The teachers should be interested in 
improving present learning 
environments. 

•The teachers should be aware of likely 
student problems in learning the 
language. 

• The teachers do not need any previous 
experience with computers and 
interactive video. 

• Teachers should come from a wide 
geographical distribution. 

Organization of design sessions is critical. 
•A typical design session is one week. 

• Several groups will usually work 
simultaneously. 

• Designers should work in groups of 
about four people. 

•Designers must be present for the full 
session. 

•Written material should be available in 
advance to the designers, outlining the 
task and describing important points. 

Other participants may be useful. 

•In addition to several teachers, a design 
group might involve a linguist, who can 
supply broader language information, 
useful for anticipating, categorizing, 
and analyzing likely answers the 
students may give. 

• A person already experienced in 



i , 305 300 



designing for Interactive media can 
stimulate the imaginative use of the 
media, and guide teachers In the use of 
highly interactive strategies. 

• No programmers should be involved to 
the design groups, as the discussion 
should focus on pedagogy. 

Designers need to understand something 
about group dynamics. 

• Teachers will have little experience with 
such Intense group work. 

•The week's work should start with a 
brief introduction to group dynamics. 

• Designers should expect conflict within 
the group. 

• Designers should try to reach 
compromises quickly. 

• Designers should not be distracted by 
technical details. 

A design session produces a pedagogical 
design document. 

•Designers make ALL pedagogical 
decisions. For a typical exercise, the 
designers should specify: 

- the video excerpt it uses, and 
whether sound, video, or both are to 
be used; 

question or instruction given to the 
student; 

- all additional sound needed to both 
languages; 

- all special facilities (such as video 
translation) 

- all pictures (photographs, graphics) 
to be shown 

nature of exercise, Including how 
student is to answer (or otherwise 
take part) 

- anticipated answer categories; 

- responses, help sequences for each 
category, paths to take, further video 
to use, etc. 

• Designers need a methodology for 
recording all results. The methodology 
should focus on interaction. The format 
of the design we use is the script, a 
technique which has been in use at Irvine 
for over twenty years. It diagrams the 
flow of the whole lesson to pedagogical 
terms, creating a specification which is 
easily Implemented by a programmer. 
The notion is to have a tactic which the 
teachers can pick up very quickly, and 
which they think of as the process of 
working with individual students. Because 
of the multiple tracks and complicated 
help sequence loops, an overview flow 



chart should be recorded at the same 
time, to indicate the connections among 
the exercises and between tracks. 

FINAL COMMENTS 

This paper has discussed: 

• The need for high quality interactions to 
language learning. 

• Strategies for obtaining high quality 
interaction. 

• A design process that leads to highly 
Interactive learning material. 

The strategies recommended to this paper 
have all been used for our Understanding 
Spoken Japanese system lYoshii, 19911. This 
highly Interactive system promotes the 
student's conversational ability through 
exposure to genuine 

conversational Japanese. Comprehension of 
the whole video is a possible goal, but not 
mandatory. 

The target audience is American high school 
and college students. Business or industry 
may also find It useful. The system assumes 
no particular text or other reference works. 
It can best be described as an independent, 
individualized supplement to other types of 
instruction. 

Ten lessons, taking about twenty hours of 
student time, have now been created. The 
development of a second ten lessons for more 
advanced students is beginning. This project 
is one of the most substantial collections of 
highly Interactive video material so far 
produced, to any subject 
area. 

We are planning to design and Implement a 
system for teaching English articles to 
students of English as a second language. 
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Abstract 

The presentation describes and demonstrates 
methods of accessing databases from an IBM 
InfoWlndow computer-based video Instruction 
(CBVI) program. "C" language procedures are 
used with IBM's IWPS authoring system, an 
IBM PS/2 Model 70 microcomputer with 
InfoWlndow touch display and a Pioneer LD- 
V6200A laser disc player. Segments of a CBVI 
program are used to illustrate database access 
(for example, learners' demographic data) and 
storage of data for future use. 

Introduction 

The paper describes methods of accessing 
databases from an IBM InfoWlndow computer- 
based video instruction (CBVI) program. "C" 
language procedures are used. The Impetus for 
the topic began In 1989 when a computer-based 
video instruction (CBVI) program was 
developed for a research project on learner 
control options (Hassett, 1991). 

Phillips' work supports the Implementation of 
tracking students' progress for CBVI/DB (1991. 
pp. 12. 41). Therefore, following the project, it 
was determined that problems with accessing 
databases needed to be addressed if evaluation 
of Individuals' learning was to be readily 
attainable for the user. Terms used are defined 
as follows: author is the creator of the CBVI 
program; user is the Instructor/trainer; learner 
is the student/employee; presentation is a CBVI 
program; application is multiple presentations 
linked together; events are segments in a 
presentation; elements are actions in an event; 
and CBVI/DB is a computer-based video 
Instruction pgram with a relational database. 

The project presentation on spiritual care of 
the patient had been authored using IBM's 
InfoWlndow Presentation System (IWPS ). 
Level 05 1A (International Business Machines 
Corporation [IBM1. 1989a; IBM. 1989b). 
Computer-based research tools were included 
as part of the instructional design (Hassett, 
1990). Four problems with data access surfaced 
while using the project presentation: a) 
learners were asked to input demographic data 
items that existed in on-site information 



systems; b) data files were created for 
individual learners and could not be easily 
aggregated; c) modifying presentation data was 
time consuming (e.g.. changing pretest items); 
and d) data required for statistical analyses 
had to be manually gathered. 

Because of project time constraints (eight 
weeks for creation of complete presentation), 
individual learners' data was printed as stored 
In IWPS log (trace activity) files. This data was 
then tabulated manually and input to a 
spreadsheet or sent for data analysis via 
electronic files created from the tabulations. 
The frustration of manually tabulating pages 
of data, much of which were stored 
electronically, prompted an in-depth look at 
the four problems. 

To resolve the immediate problem of 
aggregating learner data, a standalone "C" 
program was written to extract data from the 
IWPS system files (variable and variable 
Index). The "C" program was called by the 
presentation at the end of each learner session 
storing the data in a separate aggregate file. 
This solved only one problem and additional 
file conversion was still required. A more 
comprehensive solution was needed. 
Possibilities identified were to: a) download 
learners' demographic data from the on-site 
information system into character- delimited 
files using a standard data base management 
system (DBMS) format, such as dBASE IV; b) 
store CBVI data in an accessible format, using a 
traditional or conventional DBMS, and 
thereby facilitate statistical analyses; c) 
aggregate learner demographic data and CBVI 
data as needed, via traditional or conventional 
DBMS; and d) store test items and other 
presentation data In character format in a file, 
rather than using storage in the presentation 
graphics library. 

It was decided that the traditional file 
processing approach would be abandoned in 
favor of a formal database approach. The 
presentation was completely redesigned to 
accommodate the new database requirements. 
A rapid prototyping approach was used 
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(Hassett, forthcoming'. This demonstration 
will show the results of that endeavor. 
Database Organization of data in a database is 
crucial to ease of maintaining integrity of data 
and ease of use. Databases are integrated 
collections of automated data files related to 
one another to support a common purpose 
(Stevens, 1991, p. 8). They include what are 
commonly known as metadata and structure. 
Database management systems (DBMSs) 
provide the capability to manage and 
manipulate the data; standard query languages 
can access the data. 

All real world things are related to other things 
by relationships. The ability to represent 
things and maintain the relationships 
between/among these things is an important 
feature of a database system. Database systems 
can be based on a number of database models. 
Most agree that the third normal form (3NF) 
relational model is the model that one should 
aim for, to provide a system that has a good 
maintenance profile. Translation from this 
model to any working DBMS is easily possible. 

Relational Model 

There are three major advantages to the 
relational approach. First, compared to other 
database models, only the relational model is 
based on certain mathematical disciplines: set 
theory and predicate logic (Date, 1990, p. 13). 
fhe relational model requires that the 
relations be normalized. There is a solid body 
of theory (dependency theory) to support this 
process (Date, 1990, p. 36 ). Databases that are 
based on normalized relations (i.e., 3NF) 
guarantee against certain types of deletion, 
insertion and update anomalies. Second, 
relational databases provide a simple user- 
interface. Files are perceived by the user as 
tables (and nothing but tables); and tables are 
very simple and familiar structures. Relational 
operators such as the Structured Query 
Language (SQL) SELECT and UPDATE 
operators allow users to simply specify what 
they want to do, without knowing how it has to 
be done at the programming level. Third, the 
relational model is widely used and is well 
supported by vendors. 

In a relational database, things and 
relationship are both represented by flat files. 
Thus, relational database files have no parent 
files and no children files. Each record in a file 
is uniquely described by a key consisting of one 
or more data elements . Files in a relational 
database system are related by using data 
elements. For example, a person's social 
security number in one file may be linked to 



another file that also has that data element. In 
this demonstration, a relational database 
designed to support the CBVI system will be 
used. That database is in 3NF. 

Tools 

Most CBVI authoring languages provide some 
means of storing data. They do not, however, 
tend to provide the author or user with 
database support. This is true of IWPS, which 
requires a programmer to write the code 
necessary to access databases. dBASE IV (a 
traditional or conventional DBMS) was chosen 
for this demonstration for three reasons: it 
supports the relational model, it provides a 
subset of SQL and it is widely available. dBASE 
IV is not necessarily the best DBMS but is the 
most popular. Most microcomputer 
applications software, such as a spreadsheet, 
will Import and export dBASE IV files. SQL is 
an English-like database language that 
operates on data entirely as logical sets called 
relations, or tables (Ashton- Tate Corporation, 
1988, p. 7-1; Learning SQL, 1991, p. 1). SQL is a 
standard for mainframe, minicomputer and 
microcomputer database system programs. 
SQL is intended to be used by audiences ranging 
from computer professionals to end users. A 
number of popular DBMS software packages 
now include the SQL language. 

Design Goals 

The following goals were determined for the 
demonstration database: -store pertinent data 
in an accessible format, -minimize redundant 
data, -minimize the number of relations (files), 
and -normalize relations (i.e., 3NF). The first 
goal is obvious; that is, one must have the 
capability of storing all pertinent data in the 
database. The demonstration uses dBASE IV (a 
traditional or conventional DBMS) for 
accessibility. Some data must be extracted 
from the on-line information system and 
stored. The second and third goals eliminate 
redundant data and keep the number of 
relations (files) in the database to a minimum. 
The fourth goal normalizes relations; this 
minimizes update and deletion problems. The 
demonstration has six relations. IBM 
InfoWlndow and IWPS Computer-based video 
instruction (CBVI) is often referred to as 
interactive video. Miller, Reeve-Dusenberry 
and Sayers defne Interactive video as follows: 
The fusion of video and computer technology: a 
video program and a computer program 
running in tandem under the control of the 
learner. In interactive video, the user's actions, 
choices, and decisions genuinely affect the way 
in which the program unfolds (1988). 
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CBVI programs are commonly Interactive only 
with the learner. The author of the CBVI 
program defines the values to be used and/or 
generated by the program in terms of 
interaction with the learner. [The term values 
is used, rather than data, because the user does 
not have access to "values" in the CBVI 
program.] 

CBVI is used in the context of other systems 
that use and produce data that are sensitive to 
the presentation (context-sensitive). An IBM 
PS/2 Model 70 microcomputer is used for the 
demonstration, with an IBM InfoWindow touch 
display and a Pioneer LD-V6200A laser disc 
playek . The IWPS Version 1.0 Level 51a was 
upgraded to Level 56 for this demonstration. 
Level 56 provides a number of new features, 
including support for IBM's M-Motion. 

IWPS Editor and Interpreter 

The IWPS Editor, used to author CBVI, is 
application independent. That is, the author 
has considerable flexibility in developing 
presentations. However, the user's control over 
data used and generated by the presentation is 
limited. The Editor is used to develop (author) 
interactive videodisc presentations (IBM, 
1989a). 



The Interpreter (InfoWindow Presentation 
Interpreter IIPI1) is used for developing and 
running presentations (IBM, 1989b). It is a 
runtime standalone program. When editing a 
presentation, authors have access to 
programming languages through: (a) user exits, 
(b) IPI "hook routines", or (c) calls to separately 
executable programs. Programming modules or 
programs must be coded and compiled. User 
exits and IPI "hook routines" must be linked to 
the IWPS Interpreter prior to editing. Several 
languages are supported by IWPS. Microsoft "C" 
is used exclusively for this demonstration. 

IWPS "C" Program Language Support 

User procedures written in "C" language can 
access selected IWPS system files (IBM, 1989). 
After the project, therefore, "C " language 
procedures were written and tested (Harbison & 
Steele, 1991; Microsoft C: 1990a, 1990b, 1990c). 
The term "procedure" is used in its generic 
sense. "C" is a procedure-oriented language that 
has all the constructs associated with 
structured programming; for example, if-then- 
else (Pugh, 1985, p. 1). In "C", however, a 
function is the equivalent to a subroutine or 
function in Fortran, or a function or procedure 
in Pascal (Kemighan & Ritchie, 1988, p. 24). 
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Figure 1. IWPS Presentation with User Exit. 



X" procedures made it possible to retrieve and 
store data In traditional or conventional 
database format. These procedures will be 
shared during the demonstration and 
illustrations will be provided via segments of 
the demonstration CBVI/DB program. A CD- 
ROM programmer's library was used for 
reference and sample code programming 
support (Microsoft programmer's library, 
1991). SQL was used for relational database 
manipulation and control (Learning SQL, 
1991). 

User exits. User Exits provide variability 
during the execution of an event. A library of 
"C" functions is provided to use as calls from 
user procedures (IBM MultiMedia Solutions, 
1990, pp. 23-45). Variables defined in the 
presentation during development can be 
accessed at runtime using "C" functions. The "C" 
functions used to Lead and write to database 
files were: ipi_read_var(), ipi_update_var(), 
and ipi_set_ind(). These functions are coded in 
"C" modules and linked to the Interpreter. 
Figure 1 depicts an IWPS diagram for a user 
exit: the learner inputs her learner ID on the 
InfoWlndow touch display and the IPI accesses 
her first name via "C" procedures and displays 
her name in a hello screen. 



IPI hooks. The IPI "Hook Routines" are pre- 
defined "C" function names that are called by 
the Interpreter at appropriate times during the 
running of the presentation. For example, the 
event_start() function will cause a given action 
to take place at the beginning of every event In 
the presentation. These functions are coded in 
"C" modules and linked to the Interpreter. 

Standalone programs. Separate programs can 
be written and called by the IWPS Interpreter. 
When such a program is called, control of the 
presentation is given to that program. When 
that program terminates, control is returned to 
the Interpreter. These programs do not have 
access to presentation variables but can access 
data automatically stored in files by the 
Interpreter. 

CBVI/DB Application 

The environment in which a CBVI program is 
used determines what kind of database access 
can be provided. Both sophistication of the user 
and availability of technology are important 
factors for a CBVI/DB application. 

A team approach was used to develop the 
CBVI/DB demonstration application. The basic 
requirements were defined. Then, in parallel, 



the Programmer/Analyst developed the 
database system while the Instructional 
Designer /Author rapid prototyped the 
presentations. The team approach was 
necessary because the variable names and 
types used by the programmer in the user exits 
were first declared by the author In the 
presentation events and vice versa. The goal 
was the final application; however, at the day- 
to-day level there was a complex set of 
interwoven factors to manage: personnel, 
resources, schedules, budgets, documentation, 
and ideas (Bergman & Moore, 1990, p. 25). The 
team approach worked well. 

Requirements 

Database requirements (Figure 2) were 
identified as: (a) user demographics, (b) 
Instructional management, (c) research resou 
rce, and (d) presentation control. 

[Insert Fig. 2 about here] 

User demographics. Identification numbers are 
used for the user and for the learner. The 
Subject Information Form (SIF) tool (Hassett, 
1989c) collected five areas of learner 
demographic data In the project. This tool was 
adapted to a new, similar tool for the 
demonstration: the Learner Information Form 
(LIF). 

Instructional management. Identification 
numbers are used for the user and for the 
learner. Presentation time spent by the learner 
is stored. For example, time data can be used to 
plan scheduling of future computer laboratory 
activity. Reports are generated from the 
demonstration database; these include selected 
demographics, presentation time spent and test 
scores. 

Presentation control. The CBVI project was 
used for research on learner control options 
(Hassett. 1989d; Hassett, 1991). Changes in the 
project presentation were difficult to make. 
Therefore, each presentation for the 
demonstration is assigned an identification 
number. Time stamps are collected and stored. 
The Screening Pretest (SP) tool (Hassett, 1989b) 
can now be altered by the user. For example, SP 
items can be altered, added or deleted, and the 
number of randomly selected items can be 
changed, such as from five items to ten. 

Research resource. Data important to research 
were added to the CBVI/DB. Identification 
numbers for the user and for the learner are 
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Figure 2. Database Requirements. 



used. The four CBVI project tools are used in 
the same form or as revised: (a) the SIF tool 
(Hassett, 1989c) was revised to LIF for learner 
demographic data; (b) the Screening Pretest 
(Hassett, 1989b) for baseline learner knowledge 
prior to the CBVI; (c) the Embedded Content 
Questions (ECQ) tool for assessing knowledge 
acquisition (Hassett, 1989d); and (d) The 
computer-based ARS tool (Hassett, 1989a) for 
measuring attitudes toward the CBVI program. 
Kelly, Pascarella, Terenzlnl, and Chapman 
developed the Adjective Rating Scale (ARS), a 
public domain paper and pencil tool that 
measures attitudes toward courses and 
programs (1976). Other learner responses added 
to the database for research purposes are 
learner control choices. Therefore, examples of 
research resource activity available through 
the CBVI/DB Include: -assessment of learner 
control choices; -comparison of Screening 
Pretest baseline data with learners 1 scores on 
the ECQ; -study of learner attitudes toward 



CBVI; and -study of presentation time spent by 
learners. 

Design and Development 

The Entity-Relationship (ER) model is used In 
the database design. The ER model is closer to 
the user's perception of data and Is not meant to 
describe the way In which data will be stored in 
the computer. Figure 3 depicts an ER diagram of 
the demonstration CBVI/DB. The entities are: 
user, course, presentation, learner, test, and 
evaluation. The entity relationships and 
attributes are depicted In the diagram (Fig. 3). 
An object-oriented database design, such as 
described by Huang and Unger (1988), will be 
used with the relational model in the future. 
Modular Approach. With the modular 
approach, an application is divided into 
multiple presentations. Each presentation 
performs a cohesive function. IWPS supports 
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Figure 3. Entity-Relationship (ER) Diagram for a CBVI/DB. 



this approach by allowing a presentation to be 
called from within an event of another 
presentation (IBM, 1986a). Adding the database 
to the demonstration (i.e., the CBVI/DB) makes 
using multiple presentations simpler. The 
database provides data independence, thus 
eliminating the need to collect data from 
several program-dependent files. Figure 4 
illustrates how one may use multiple 
presentations with IWPS and a CBVI/DB. 

Defining and Querying the Database. Sample 
dBASE IV SQL statements used to create the six 
relations in the database will be demonstrated. 
The relations declared through the CREATE 
TABLE are called base relations. These 
relations will be created and stored as files by 
dBASE IV. Base relations are distinguished 
from virtual relations created through the 
CREATE VIEW statement. A virtual relation 
may or may not correspond to an actual file. 



SQL does not provide a way of specifying key 
CREATE TABLE command. This must be done 
using the CREATE INDEX command. 

The demonstration will include a sample query 
of the relations in the database. 

ToolBook (v. 1.5) will be used to build a front 
end for the CBVI/DB. This will allow the user to 
maintain data In its original form so it can 
still be accessed by other programs (Asymetrix 
Corporation, 1991, p. 19). The ToolBook front 
end (book) will be used to demonstrate the 
flexibility of the database. 

Implementation 

A Typical Learner Session and a Typical User 
Session will be demonstrated, based upon the 
CBVI/DB depicted in Figure 4. dBASE IV will be 
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used as the DBMS. The demonstration will 
show an application that uses procedures 
written In M C" that provides access to relational 
database files. The demonstration will 
illustrate database access, report generation 
and storage of data for future use. 

Conclusions 

Users need automated access to learners' data. 
Problems with accessing databases in CBVI 
were addressed. Methods of accessing databases 
from an IBM InfoWindow computer-based 
video instruction (CBVI) program using "C" 
language procedures were described: a 
CBVI/DB. Later, object oriented design (Huang 
& Unger, 1988) and statistics capability should 
be added to the applications. 

Researchers have investigated prediction of 
success in courses. For example, Becker and 
Unger's study created and validated a tool to 



predict success in an introductory 
programming course (1983). A tool such as this 
could be computer-based and used to predict 
success prior to learners 1 initiating a set of 
CBVI/DB presentations, for a given course. 
Further research is needed in this area. The 
CBVI/DB will be tested in research 
environments. 

The use of databases with CBVI ought to be 
promoted to increase the types of use that 
involve evaluation of learners* responses. An 
"open architecture 11 philosophy is needed to 
make this work easily portable to other 
environments. More flexibility should be 
provided to users about presentation control 
data. More research is needed to better 
understand what the next generation of 
programs should look like. User demographic 
data is important for research, instructional 
management and presentation control. 
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Providing easy access to the data necessary for 
course management should be an Integral part 
of the CBVI design (I.e., the creation of CBV 
I/DB). 
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Abstract 

Many theories hr f e been developed about 
learning and retention. In this paper we shall 
discuss an independently arrived at synthesis 
of existing learning theories embedded in an 
original Intermediate Hebrew Computer 
Assisted Instruction (LAV /CAI) project. The 
latter was the outcome of a cooperative 
agreement between the U.S. Government and 
Brigham Young University. Elaboration 
theory, a modern instructional development 
strategy which uses existing cognitive 
principles, will be referred to in connection 
with the structural format of the IAV/CAI 
project. 

Points of contact will be discussed between 
many of the theoretical issues and their 
practical application to real problems in CBT 
development. Several diagrams and flowcharts 
are included to simulate some of the actual 
computer interface screens and to graphically 
illustrate the nature of the lesson design. 

Introduction 

With the advent of technology, learning has 
taken a new twist. Courses no longer need to be 
developed strictly on paper. Computer Assisted 
Instruction (CAI) is a multidimensional, 
fascinating medium which, when used 
properly, has been known to enhance student 
motivation and desire to stay with the course. 
We believe that a design based on a synthesis of 
modern learning theories can enhance long- 
term retention of the material. 

Below, we will describe an original Interactive 
Video Computer Assisted Instruction (IAV/CAI) 
Hebrew language courseware package, 
highlighting computer learning interface 
features and the language-learning 
methodology used. Finally, using the 
instructional design strategy known as 
Elaboration Theory, we will show how 
computer-assisted instruction based on an 
independently arrived at synthesis of modern 
learning theories can create a state-of-the-art 
language learning tool. / 

First, the courseware package. In 1982 Brigham 
Young University (BYU) and the U.S. 



Department of Defense collaborated to produce 
three interactive videodisc language 
Instruction projects. This interactive video was 
achieved through video materials that were 
mastered on laser disc and incorporated in 
computer assisted instruction (CAI) courseware 
(via video disc player) for the learner to 
interact with. One side of a laser disc holds 
54,000 frames or one half hour of video 
material. Each laser disc has two audio tracks. 
In the Hebrew language project we used track 
two for the recording of a more carefully 
enunciated audio (MCEA) mode. 

The programming was done at BYU, while the 
subject matter experts (SMEs) from the 
Department of Defense designed and developed 
every aspect of the course, including the 
selection of video materials and their 
copyright releases, the recording of the MCEA, 
the structure and plan of each lesson and the 
actual coding of hundreds of screen design 
sheets (Knisbacher, 1987). All these projects 
consisted of an initial prototype course and a 
follow-on, more sophisticated "phase II." All 
ihe packages were intended for language 
learning. The entire project took six years to 
complete. The authoring tool was Courseware 
Design System (CDS) with Courseware Design 
Language (CDL) for the fine-tuning. A video 
overlay board (the MIC 2000 by Learncom) was 
used to project the video on the same computer 
screen used to present lesson text and graphics. 
A Zenith AT 248 (an upgrade from the 
prototype) was the master engine driving the 
process. 

In phase II of the intermediate level colloquial 
Hebrew courseware, another component was 
added, the Instavox (from the Latin for "instant 
voice"). The latter is a rapid access analog audio 
device that was used for the recording of 
cultural notes, idioms, expressions, and the 
Advanced Level Total Comprehension Test, all 
on a rather clumsy 14" floppy disk. Today's 
digital technology is far more compact and 
transportable (see diagram on figure 1 ). 
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Language Acquisition and IAV/CAI Design 
Aspects Designing IAV/CAI is not a simple 
endeavor: It is a multifaceted process. First, 
there is the actual CAI aspect, which is quite 
different from normal platform instruction 
development. One cannot just take a paper 
course and encode it into the computer. It is too 
expensive and time-consuming a medium to be 
just a page turner. It should contain ample 
remediation material and multiple branching 
flexibility. Another essential consideration is 
the aesthetic appearance of each screen (page) 
presented to the learner. Since reading a 
computer screen is more difficult than a 
printed page, care should be taken not to 
overcrowd it, to use upper and lower case letters 
and to leave ample space between lines of text. 
Related pieces of information (page numbers, 
content title, student options menu, feedback, 
etc.), must appear in their allocated place on 
the screen cosistently. And finally, in addition 
to screen color selection, there is the language 
aspect and related methodology. 

A CAI course can be designed to supplement 
existing, regular classroom instruction, or to 
support self-paced learning (in stand-alone 
mode). The addition of video to CAI opens up 
new avenues for the exploration of CAI -based 
teaching effectiveness. "A picture is worth 
1,000 words" is more than a cliche. Video not 
only makes learning livelier, but also clarifies 
and puts the point across faster. This is 
especially true of language learning, where a 
gesture or facial expression may help a person 
understand the context even without knowing 
the meaning of each individual word. 

Dr. Janet Murray (1989) author of the Athena 
language project at MIT, sums up the benefit of 
interactive video in the teaching of foreign 
language as follows: M Interactive video can 
provide the student with a sense of immersion 
that no other technology can duplicate, as the 
students are surrounded by a multiplicity of 
language sources-video, stills, audio, graphics, 
texts, -allowing them to see, hear, and read the 
language simultaneously. This variety of 
language as it exists within the visual, social, 
and cultural world, allows for immersion in 
the truest sense of the word/ 1 

Phase II of the Hebrew IAV/CAI is strictly an 
Intermediate-Level Colloquial Hebrew course 
and can be used as enhancement to existing 
platform instruction, or in stand-alone mode. 
It is based on a sitcom from the Israeli TV 
series "Close Relatives," somewhat comparable 
to our "Family Ties." Only comedy was used 
here because it is so closely tied to colloquial 



language. 

Contextual methodology is the language 
instruction approach of choice in this project. 
Current language instruction theory focuses on 
real language (as opposed to the made-up 
sentences that one finds in so many text 
books). In keeping with that idea, everything in 
Phase II is prompted by actual usage in the 
video, i.e., by real live language. The learner is 
encouraged to listen to a whole string of 
vocabulary items in context, rather than to 
focus on a single word. The advantage ofhis 
approach should be obvious. 

Our overall computer-assisted foreign language 
acquisition philosophy is very much in tune 
with the thinking of noted educator Michel 
DeBloois (1983) who states that courseware 
authoring for videodisc requires 
"multidimensional thinking and advanced 
design tools." In attempting to provide as many 
tools as possible to enhance the learning 
experience, several features were programmed 
and incorporated into the project. These 
include, among others, the ability to watch the 
video and/or listen to an utterance as often as 
the learner wishes, using both audio tracks 
alternately. Where problems remain, a 
transcript of the audio can be requested along 
with the English translation and/or of the 
Hebrew-English vocabulary list (word bank). 
All of these are always available from a student 
options menu at the touch of a function key to 
make the program as user-friendly as possible. 
Of course, backing out of any activity 
whatsoever and switching to another is a 
constant option. There are very few "system 
control" features, i.e., instructor-controlled 
activities. For example, the learner may not be 
allowed to answer a question more than once 
while in "test mode" or may be encouraged to 
complete a transcription exercise prior to 
having access to the entire text. These 
restrictions, however, are few and far between. 
Rather, "student control" predominates, thus 
making the learner master of his* own destiny. 

Elaboration Theory Having talked about our 
hardware configuration and language 
acquisition philosophy, let us look at the 
overall design strategy and the tools used to 
Implement it. 

Elaboration theory is a macrolevel strategy 
and structure for the development of 
instructional design. Based upon the work of 
Reigeluth and colleagues (1979, 1980, 1982, 
1983, etc.), it advocates that instruction be 
developed the way knowledge is arranged in 



memory, i.e., hierarchically. 

Elaboration theory embraces well-established 
cognitive approaches to learning and 
memorization a la Gagne. However, 
elaboration theory recognizes other 
relationships among the various content types. 
Although it is relatively new, elaboration 
theory shows marked increase in the ability to 
learn concepts, which in turn, promotes better 
transfer of concepts and their classification, 
leading to the application level. Elaboration 
theory begins at the application level, 
proceeding down and back to the entry point at 
the top. 

Elaboration theory and Norman's Web theory 
(1983) have a common denominator: both hold 
that "memory is a network of Interrelated id 
eas" leading to the duplication of the theory in 
practice. 

♦Note that for the purpose of simplicity, the 
learner will be referred to as 'lie" or ,, him. ,, 

The instructional design of elaboration theory 
advocates the need to: 

a. state the overall ideas within the general 
content (procedures, concepts or principles) 

b. use epitomes to present the simplest, most 
essential ideas to show their application in 
the larger picture 

c. relate these ideas to things familiar to the 
student through previous learning (use of 
analogies) 

d. break up the general content into more 
elaborate detail (from the general to the 
particular) 

e. repeat the above process in reverse-from the 
detailed to the general 

f. let the learner determine the sequence of 
information 

Lesson strategies according to elaborative 
theory should: 

g. present ideas in simple to complex sequence, 
after presenting the general structure 

h. use Gagne's prerequisite knowledge, i.e., 
always teach those behaviors that become a 
foundation for the rest of the content 

i. use summaries, practices items and 
examples 

J. synthesize instruction at the end of the 

lesson by relating new ideas to previously 

learned information 
k.make instruction relevant through the use 

of analogies (relates to previous learner 

experience) 

1. provide hints on chunking, organization, 
synonyms and antonyms in order to teach 



cognitive strategies (this car be included as 
part of the overall instructional strategy) 
m.let the learner control the mode, sequence, 
and learning strategy of Instruction 

Reigeluth has used the analogy of the "zoom 
lens in a camera." He suggested first presenting 
the learner with the overall picture (without 
going into detail), then zooming in by breaking 
the large picture Into its component parts and 
relating them to one another. Finally, 
summarizers and synthesizers are used to 
move the learner to the general picture, once 
again. 

If all of these steps are followed, the 
assumption is that learners will develop better 
understanding of the materials, gain more 
stable knowledge and thus strengthen their 
long-term memory. 

In conclusion, the designer should organize the 
learning material into groups of related 
tasks/concepts/ideas, establishing a 
relationship amongst the various content 
items, which are then arranged into 
hierarchies with the lesson's epitome at the 
top. Summarizers and synthesizers are 
arranged to reflect the exact content 
interrelationship and, finally, cognitive 
strategies and their placement within the 
general scheme of things. 

While elaboration theory seems to inform the 
overall philosophy used for the instructional 
design and development of the Hebrew IAV/ 
CAI, Operant Conditioning (Skinner's learning 
machine) is also represented, i.e., active 
learner participation , feedback and 
reinforcement, presentation of small segments 
using the "building block" approach, etc. 

How does all this tie Into the IAV/CAI project? 

Video Disc-Level Activities After a thorough 
content analysis based on elaboration theory 
principles was conducted, the overall content 
was arranged hierarchically, and all the 
concepts and principles were identified. A 
three-way division was decided upon 
consisting of disc, lesson and segment level 
materials. Within each of the above, 
prerequisites precede more complex structures, 
becoming the foundation for new learning and 
the glue for cumulative learning. 

This Disc-Level Activities portion of the course 
presents the general structure, as per strategies 
suggested by elaboration theory. At this level 
there is no specific detail. The intent is to give 
the learner an overview, a point of departure, 
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before the smaller parts are presented In the 
next level. 

Within disc side one of phase n (see Course Map 
figure 2) is a completely fleshed-out course, 
consisting of seven lessons which break down 
into twenty-four segments that we shall 
describe below (see Disc Level Menu figure 3). 
These amount to 160 hours of 
student/computer time. 

Typically, the constraints of normal platform 
Instruction make it very difficult to address a 
variety of entry levels. The computer is an 
excellent tool for solving that problem. There 
are three distinct entry levels in the course, 
each interrelating with the other, each with a 
stand-alone capability. Less confident students 
may wish to enter the course at lesson or 
segment-level. On the other hand, the very 
strong student may wish to learn at disc-level. 

A newcomer to the program win most likely 
want to see the Glossary. Here he is introduced 
to the sitcom actors, an advanced grammar 
organizer, a complete listing of idioms with 
their English translation, along with culture 
notes and their translation; all this, lesson-by- 
lesson. In fact, the grammar, though an 
Integral part of the course, can be accessed as a 



Disc-Level Menu 

■ Disc Video 

■ Disc Video (with culture notes) 

■ Disc Video (with idioms) 

■ Glossary (advanced organizer) 

■ Diagnostic Test 

■ Disc-Level Final Test 

■ Advanced Level 

Total comprehension Test 

■ Lesson-Level Materials 

■ Grammar Module* 

■ Help 

■ Quit 



separate entity to be used as reinforcement for 
a grammar review course. 

At this point, the learner will be encouraged to 
see the "K* picture," namely, to watch the 
entire hall hour of video uninterrupted . He 
may repeat this effort with cultural notes 
and/ or idiomatic expressions. 

Now, in the best tradition of CAI, the learner 
has. among other options, the choice of taking 
a diagnostic test to see if he is ready to take the 
final test and bypass this disc side. If he passes 
that test with a score of 90% or better, he 
qualifies for the final exam without further 
ado. If he then passes the final with a score of 
80% or better, he gets credit for the course and 
may continue with another disc side. The 
diagnostic test requirement Is deliberately 
higher than the final, since most learners will 
benefit from the Intensive review provided in 
the body of the course. 

If, after viewing the half-hour comedy, the 
learner takes the diagnostic test and fails, the 
Computer Managed Instruction (CMI) "advisor" 
will instruct him to drop down to at least 
lesson level and suggest what lessons to do on 
the basis of questions missed. This approach 
has been claimed to be more effective than 
unmediated learner control. 

Cognitive psychology stresses the importance 
of making connections between previous 
knowledge and new learning. It Is easier to 
adapt learning to learner needs when the 
learner's background is well-known than when 
a randomly formed group Is involved and 
learner backgrounds are not well-defined. 
Adaptive learning programs could help solve 
the problem, but they are too costly to develop. 
However, although learner-control CAI is more 
cost-effective than adaptive programs, one does 
not know if the average learner possesses the 
know-how to select the proper remediation 
materials for his particular needs. 
Specifically, when a learning task is 
particularly difficult, the learner may simply 
opt to get out and go on to another task. Several 
studies conducted on the subJect-Hanse, Ross & 
Rakow (1977). Tennyson & Rothen (1977). 
Steinberg (1977). Hannafln (1984). and Ross & 
Morrison (1989) -have discovered that 
"uncontrolled" learner-control, does not 
Improve performance. 



Figure 3 
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Figure 4 
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Student Options: 




video audio (in context) video (in context) Instavox F7(EX1T) 
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Lesson-Level Menu 

M Lesson Video 

■ Lesson Video (with culture notes) 

■ Lesson Video (with idioms) 

■ Diagnostic Test 

■ Lesson-Level Final Test 

■ Lesson-Level Material 

■ Lesson Segments 

• Segment one 

• Segment two 

• Segment three 

• Segment four 

■ Word Games 

■ Final Test 

■ Help 

■ Quit 

Figure 6 

In a research study investigating two 
instructional design variables related to 
concept learning, Tennyson (1980) suggests 
computer- managed advisement as 
remediation for some of the learner-controlled 
instruction strategies. Results showed that 
students under course advisement performed 
better than those under strict learner control 
conditions, and more efficiently and timely 
than the adaptive control group. 

Our design puts great emphasis on learner 
navigation, preference and control. However, if 
the approach does not agree with every learner, 
or if, due to this approach, a learner cannot 
pass the test, CMI will step in to advise him as 
to the best avenue to pursue. 

If upon failing the disc level diagnostic test, the 
learner decides to change his approach 
completely and instead of a top to bottom 
approach, go directly to segment level, the 
system is, flexible enough to allow this to 
happen. 



In addition, to challenge the advanced student, 
there is an advanced-level total listening 
comprehension test available on Instavox. 
This is a recorded listening comprehension test 
consisting of an analysis of the sitcom and its 
actors by male and female native speakers. The 
initial recording was done without a script (to 
maintain language spontaneity) in rapid-fire 
Hebrew, dialogue style. Because opinion is 
involved, the language level here is at least 
three plus (editorial-type, on the government 
scale). A series of test items follows each 
recorded segment. The directions for the test 
are also 

aural. The learner is instructed in the target 
language to select or type in an answer as the 
need arises. 

It should be noted here that all tests (lesson and 
disc level) are automatically graded and the 
scores maintained for the instructor . The 
computer also keeps tabs on how many times 
the student attempts each question of the 
segment level exercises (not graded so as to be 
non-threatening). In other words, phase II goes 
beyond CAI into the realm of CMI. 

The learner may request assistance at any 
level. He can review, skip or terminate the 
video at will. There is variable access to the 
cultural and idiomatic expressions. If the 
learner so chooses, he may have the video stop 
automatically at each point where a 
commented cultural point or idiomatic 
expression occurs. The screen will shrink into 
a window (see figures 4 and 5) zooming in on the 
face of the speaker. Outside the window, the text 
of the comment will appear, and then the 
comment will be heard. In fact there are two 
different windows, a round one for the 
idiomatic expressions and a square one for the 
cultural notes. After requesting and obtaining 
the necessary assistance, the learner is 
returned automatically to the starting point 
where HELP was first requested. 

Leston-Level Activities At lesson level the 
learner can repeat the above process by first 
viewing the lesson video portion with all the 
trimmings (see lesson level menu figure 6). 
Once again, he may take the diagnostic and, if 
he passes it with an acceptable score, he may go 
on to take the lesson-final test and continue 
with the next lesson. All the lessons use 
epitomes and tie into one another and into the 
larger structure or picture. The examples 
chosen to illustrate a point are usually related 
to scenes familiar to the learner (analogies) in 
order to facilitate comprehension and transfer. 



Here the general thrust is from the general to 
the particular, and back to the general. The 
student Is encouraged to participate, creating 
an atmosphere of active learning. Ample 
feedback exists along with hints and guidance 
to help the learner, while minimizing the 
frustration level. HELP and QUIT function 
keys can be put to good use at any time and 
from any point in the program. 

If the learner falls any diagnostic test at lesson 
level, he is instructed to drop down to segment 
level and do those activities relevant to the 
missed portion of the test before taking the 
final exam. In fact, If at disc level the learner 
attempted to take that level's diagnostic and 
was dropped to lesson level, he must pass every 
lesson-level final before attempting to take the 
disc-level final test. The reasoning here is that 
lesson level finals are used as progress tests; 
since the overall final test is comprehensive, it 
is Important for the student to show readiness 
to continue on. 

Word review is another feature built into each 
lesson at menu level. The learner may get here 
either after having completed all the segments 
of a lesson, or after or before taking the lesson 
test. Since these games are not graded except 
through CMI control, it is a suggested exercise, 
but not a mandatory one. The review is in 
computer-games format (varying from lesson 
to lesson). There is a concentration game, (a 
classlfy-type game), and listening to the 
Hebrew track to find a match to an English 
utterance projected on the screen (all done 
contextually). 

Segment-Level Activities. Segment-level 
activities all relate to the short video cuts at 
that level. In other words, the half hour sitcom 
was divided first into contextual portions 2-3 
minutes long, which we called lessons. 
Following that, each lesson was divided further 
into so-called segments—small, palatable 
contextual portions, lasting anywhere from 30 
seconds to a little over one minute. Although 
the design is flexible, the approach at each level 
of instruction is always from the general to the 
simplest idea represented, and then back up to 
the most complex. The segment level is the 
most elaborate of the three levels, taking the 
learner on a voyage through the intricacies of 
Modern Hebrew colloquial language. IAV/CAI 
design principles dictate the need for 
elaborating on a small segment of video at a 
time. This approach satisfies the criteria of 
Skinner's learning machine, i.e., presenting 
the student with one small portion at a time 
and using each preceding block as antecedent 



learning for the next, etc. Gagne's (1984) 
learning hierarchies approach is used 
throughout, with every level of instruction a 
prerequisite to the next. 

There are no tests at this level; however, lesson 
level tests cover every bit of what was learned 
at segment -level. Here again, a very close 
relationship exists both horizontally and 
vertically among the various levels of 
instruction. 

Pribram (1962) believes that reinforcement 
should be structured to fit the learning 
situation. This program allows the learner to 
structure reinforcement to fit his learning 
style. While in learning mode at segment level, 
the student can view the video in any of the 
aforementioned ways, or request to see a 
written Hebrew transcript of the audio track 
(see segment-level menu). Here, too, the learner 
can listen to the whole transcript, or to a single 
utterance in either normal mode or MCEA. 
From this screen there is also access to the 
English transcript, word bank, idioms and 
culture notes (see figure 7). with or without 
video, or to linguistic tutorials. The latter deal 
with grammatically irregular vocabulary 
items, all in context and through analogies. A 
distinction and cross-comparison is made 
between colloquial language and 
grammatically correct language (roughly, the 
linguistic distinction between descriptive and 
prescriptive approaches). 

It is at segment leve\ and through the 
transcript, that the grammar review tutorials 
may be accessed. Here the student will find 
three levels of learning and reinforcement. 
Again, elaboration theory is reflected. The 
grammar items are "chunked" together by 
subject, with additional examples added, for a 
built-in cognitive strategy. Every grammar 
item begins with a prediction question, 
designed to Introduce the learner to the subject 
without much detail. Defined concepts are 
introduced, and the learner is asked to 
discriminate and classify them. The program 
then shifts to a more elaborate explanation 
through analogy and examples. Additional 
practice is available at the press of a key, and 
the learner can demonstrate the rule ( a la 
Gagne) if he is ready. (See grammar rule flow 
chart figure 8) None of the above is forced on 
the learner, who can decide what he wants or 
needs to learn and what he already knows, 
before making a selection. However, it should 
be emphasized again that the tests are very 
thorough, and although an attempt is made not 
to overtest, the terminal objectives are all 
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covered. In fact, the final test is properly 
representative of every lesson-level test. 

The random access capability of the laser disc 
is particularly valuable in the various practice 
materials designed to increase comprehension. 
After viewing the entire segment video, the 
learner can choose to do exercises. During any 
exercise at segment level he may repeatedly 
watch Just that smaller portion of video that is 
contextually related to the question he is 
working on. 



Segment-Level Menu 

■ Lesson Video 

■ Segment Video 

• with vocabulary 

• with culture notes 

• with idioms 

■ Transcript 

• with audio (normal/MCEA) 

• video (full/segment) 

• with special vocabulary 

• with culture notes 

• with idioms 

• with grammar notes 
^ • English translation 

Practice Materials 
" Word Bank 
Help 
Quit 

Figure 7 

Various approaches are used to enhance the 
learning process, including prediction 
questions designed to spark curiosity, 
hypothesis learning, viewing video without the 
sound track, and listening without the video. A 
favorite with learners at all levels seems to be a 
type of prediction exercise which starts with a 
word list in the order of its appearance in the 
video. In association with multiple-choice 
questions, this helps the learner to construct 



the plot. Other exercise types are matching. 
CLOZE (Fill in the Blanks, Gagne's "verbal 
skills"), scrambled sentences (Gagne's 
"demonstrate the rule"), word chaining, 
transcription, multiple choice questions, true 
or false, gisting, indicate the speaker, etc. 

Steinberg (1984) talks about games, drills, and 
simulations. She suggests that the courseware 
designer keep the number of items short, and if 
the learners are likely to require additional 
practice on certain sets of items, there is a need 
to design a variety of formats. In this IAV/CAI 
project each of the previously listed drill types 
changes format if presented more than once. 



It should be noted that feedback is an 
important part of any CAI package. Not only is 
there extensive feedback for wrong answers- 
hints, instructions to watch or listen to a given 
segment again, etc. --but there is also positive 
reinforcement for correct answers. Not Just a 
"Nice work!" kind of comment, but an 
explanation as to why the answer is correct. 
Not every correct choice reflects knowledge on 
the part of the learner; at times it may be due to 
a lucky guess, in which case an explanation is 
due. 

With very item of the video sound track 
analyzable down to the Individual utterance, 
with tutorials on almost every aspect of the 
grammar and with exercises to reinforce every 
point related to content, the segment level 
provides remediation and glue for even the 
weakest learner, while preparing everyone to 
pass the disc-level final. 

Concluding Remarks In this article we have 
attempted to expose the reader to the endless 
possibilities of combining learning theories 
and proper design strategies with exciting 
multimedia to create intricate student- 
computer interactions. 

Interactive courseware will reflect the 
creativity of the seasoned pedagogue who 
designs it. Geoffrey R Hope, Heimy F. Taylor, 
and James P. Pusack (1984) describe computer 
assisted instruction as follows: "Many of the 
positive feelings CAI frequently engenders can 
be traced to a single factor: the computer's 
liveliness. While the screen may present 
nothing more in terms of content than a 
workbook does, by having each item pop up as 
though from nowhere, and by responding in 
some way to the student's answer, the program 
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Learning A Language Rule 



Demonstrate: "In Semitic languages (which are based on a triliteral stem), 
adding a prefix, Inilx or suffix changes the grammatical but not the lexical 
meaning.** 



i 



Demonstrate "then 
Semitic languages".. 



I 



Classify Semitic languages 
by using a definition 



i 



Discriminate languages 
by family type 



l 



Demonstrate "if adding prefix, infix, 

suffix to triliteral stem and only 
the grammatical meaning changes... 







1 




Classify lexical 
meaning by using 
a definition 








i 


Classify triliteral 
stems by using 
a definition 




Classify prefix, etc, 
by providing an 
example of each 



I 



Discriminate 
triliteral item 



Discriminate prefix, 
infix, suffix 



PREREQUISITE: College level basic Hebrew 
Figure 8 



transforms o« :erwise inert exercises into 
active materials. Language study is 
particularly well-suited to a dynamic context 
like this; some of the mind-numbing effects of 
written language exercises are changed in to 
lively and engaging qualities by the computer. 
The computer allows one-on-one interaction. 
The amount of control over events is shared 
fairly equally between student and machine: 
the computer asks the questions and has the 
answers; the student decides when to turn it on 
and off, which material to work one, and how 
fast to go. Students rarely have s::ch power over 
their teachers. Teachers rarely seem as 
patient." 

In Just a few years this technology has 
progressed so far that one can oniy Imagine 
what the future will bring. But one thing can be 
stated with fair assurance: No matter how 
much technological progress we make, we shall 
still have to come back to the "unprogrammed" 
Homo sapiens to teach computers to behave 
more intelligently and help technology take 
another leap forward. 
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DEVELOPING AN INTERACTIVE MULTIMEDIA NETWORK FOR DISTANCE 

EDUCATION 



Dent M. Rhodes and David Doss, 
Illinois State University 



Abstract 

This paper describes the present and 
anticipated development of interactive 
multimedia networks which enable users to 
share computer-based Instructional resources, 
locally and at a distance. A multimedia 
computer-based network can also enhance the 
effectiveness of interactive television 
Instruction In distance education. Both local 
area and wide area multimedia networks have 
their own specialized hardware, software, and 
operational problems. However, problems of 
system implementation and cost are less 
significant than problems of instructional 
design and preparing materials and teachers. 

Introduction 

A Increasingly common form of interactive 
multimedia instruction is one in which an 
Individual or group of students interact with 
content and technology at an "Instructional 
workstation" (Rhodes & Doss, 1989-90). 
Linking the workstations in a network 
configuration enables users to share computer- 
based instructional resources. These resources 
can be made available to teachers and students 
through a local area network (LAN) and can 
also be incorporated into a wide area network 
(WAN) for distance education. 

In an Interactive multimedia network, users at 
one workstation share hard-ware and software 
resources with users at other workstations. 
For example 41, ey can share video materials on 
tape and disc, text and visual data bases and 
compressed and full-motion video on CD-ROM, 
as well as standard and custom CAI and CBT 
programs. The media can be 1 sed for 
Individualized, self-paced instruction or as a 
aid in teacher-directed instruction. 

For distance education, interaction between 
students and teachers over a network can be 
facilitated through a telephone link with a 
minimum of added expense. When two-way 
interactive television is added to the 
instructional network, students and teachers 
can communicate with each other "face-to- 
face." 



Local Area Network Configuration 
Hardware 

In the local area network we now have 
configured at Illinois State University, the 
principal workstation/flle server consists of a 
Zenith 386-16Mhz microcomputer with 4Mb 
RAM, 40Mb hard disk, two floppy drives, VGA 
monitor and mouse. A Panasonic video 
monitor. Pioneer laser disc player, Panasonic 
VCR NEC CD-ROM player, modem, and printer 
are peripherals. A fax machine will be added as 
budget permits. 

The Panasonic VCR is connected to the micro- 
computer through an interface board and 
cabling supplied by BCD Associates, though 
future plans call for conversion to an NEC VCR 
with a serial port connection. The laser disc 
player is connected by cable to the 
microcomputer serial port . Two 8088 
microcomputers, a Columbia and a Zenith, 
each with a 640Kb RAM, VGA monitor and 
Amdek video monitor, serve as "remote" 
stations. The remotes are now connected to the 
principal workstation by twisted-pair, 
shielded cables; the video and audio 
components, by standard A/V cables. 

Software 

A windowing program, Quarterdeck System's 
DESQview with QEMM 386 (version 5.1) 
enables users to control the multiple devices. 
QEMM 386 is a memory manager which 
converts extended memory to useable expanded 
memory. Multiple windows in DESQview, each 
containing a different program, can be used to 
control devices in 

the system. For example, a CAI program which 
sets up a problem to be solved can appear In one 
window; in a second, a program which gives the 
user control over both video tape recorder and 
laser disc player; in a third, the access program 
for a CD-ROM encyclopedia. Through a time- 
sharing technique, DESQview is able to 
maintain execution of programs in all 
windows simultaneously. 

The network itself is an ARCNET Local Area 
Network. It uses a token passing bus topology 
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and LANtastic software from Artisoft, Inc. 
LANtastic is Network Basic Input Output 
System (NETBIOS) compatible. As such, it 
conforms to the Open Systems Interconnection 
(OSI) model for the transport layer, session 
layer, and presentation layer. At present, 
computer data are transmitted over a high 
grade dual twisted pair cable with shielding; 
sound and pictures are transmitted over 
standard audio and video metal cable. 

We intended to use the NETBIOS program, "The 
Network Eye," also by Artisoft, Inc., to permit 
sharing across the network of the hard drives, 
floppy drives, serial ports, parallel ports, 
modem, printer, laser disc, VCR, and CD-ROM. 
However, we have found that many of these 
peripheral devices can not be shared across the 
network with Network Eye or other standard 
LAN software. The "Network Eye" software also 
does not transport graphic images because it is 
written to require minimum RAM (30Kb for a 
master; 6Kb for a workstation). 

We have written custom programs in "C" 
language to permit a direct interface with the 
NETBIOS. In this way, screens composed of 
both text and graphics can be captured on the 
server and passed to remote stations. Keyboard 
combinations used by DESQview and 
LANtastic which create interference (such as 
those employing the "Alt" key) are also 
rearranged. NETBIOS has become the defacto 
industry standard and is Implemented in all 
major networks. Through using a standard 
interface, the custom programs can be 
transported to other networks. 

Network Operation 

Any of the computers in the local network can 
monitor and/or control (a "Master") or be 
monitored and/or controlled (a 'Workstation"). 
These terms describe network functions, not 
hardware functions: a network fileserver is a 
Master in "The Network Eye" terminology if it 
is used to monitor or control a remote 
computer. 

Monitoring and being monitored can occur 
simultaneously. For example, students at one 
computer can control peripherals at a second 
computer while being monitored by a teacher at 
a third computer. In addition to the monitor 
and control options, each Workstation still has 
the capability to perform other operations 
requiring network resources, such as printing a 
document on a network printer. 

Access to control or monitor a remote 
microcomputer is determined by a system of 
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passwords. Passwords for different levels of 
access or control are specified on the command 
line as the software at a Workstation is loaded 
and lntlalized. 

A Master can display windows that may be 
"sized" and moved anywhere on the Master 
screen. It is thus possible to view (and control) 
four Workstations simultaneously by placing a 
Workstation's screen into a window and 
moving the window to one corner of the 
Master's screen. Text can also be "cut and 
pasted" among any Workstations displayed on 
a Master's screen. These options greatly 
imSrease the possibilities for multimedia 
instruction. 

Instructional Designs 

However, conventional Instructional designs 
in the "presentation - recitation" format will 
not be sufficient to use the resources of an 
interactive multimedia instructional network. 
An alternative is teaching which emphasizes 
knowledge development rather than 
information acquisition; problem/ task- 
centered learning rather than teacher-centered 
presentations. Teacher to student and student 
to student Interactions will be "transactive," 
through and with the various media, rather 
than merely reactive (Rhodes and Azbell, 1985). 
Students may work individually or in small 
groups at an Instructional workstation instead 
of at rows of desks. 

When the interactive multimedia instructional 
network is used for individualized instruction, 
a student at a remote terminal can use any of 
the devices available at a central location. As 
an example, the courseware design will give the 
student a task to accomplish: e.g. determine the 
optimum mix of vegetables and planting 
procedures in order to maximize yield in a 
given plot. The student can then access 
information from a variety of sources at a 
central site - video tape, video disc, CD-ROM, 
CAI -- to accomplish the task. 

By monitoring the student's progress, a teacher 
at another location can provide help if tl e 
student is having difficulty or requests 
assistance. The teacher can monitor the 
student on the computer screen and 
communicate through the network, and/or 
over a telephone connection. 

For teacher-directed instruction, a teacher can 
lead individuals or small groups at remote 
terminals through an instructional sequence. 
For instance, If the students are to study a topic 
in ecology, a teacher can provide the 



information in a predetermined sequence 
through computer text, video tape, video disc, 
and/or CD-ROM. Communications with 
students can again be done through the 
network or telephone lines. 

Multimedia Interactive Television 

Conventional Interactive television 

Computer-based multimedia instruction also 
has the potential to affect significantly the 
ways interactive television may be used for 
distance education. From an instructional 
perspective, interactive television teaching 
technology for distance education today can be 
compared with the automotive industry of 100 
years ago. Inventors then were able to combine 
the technology of the time in new ways, but still 
thought of their work in conventional terms. 
Hence they developed the "steam-powered 
horseless carriage" as an innovative form of 
transportation. It was some years before they 
could design an automobile which made good 
use of the Internal combustion engine and did 
not simply reproduce the basic features of 
horse-drawn vehicles. 

In much the same way, distance educators are 
now prone to use 20th century technology to 
reproduce the school and college classroom of 
the 19th century and give the result an 
"education in the 21st century" label. The 
actual teaching itself is conventional 
classroom practice extended over a wider 
geographical area. The conception of teaching 
employed has not kept pace with the 
opportunities afforded by the technology. 

Most written descriptions and live 
demonstrations of two-way video/ audio 
television In educational applications reveal 
that Instruction retains the traditional 
"presentation - recitation" form (Czech, 1989; 
Hannu, 1990; Ward, 1990). The teacher can now 
see students on a video monitor, but the setting 
still closely resembles the traditional high 
school and college classroom. The teacher 
talks, asks questions of students, or respond to 
student questions should there be any. 
Interaction takes the form of verbal exchanges. 
This approach differs from the standard 
single-teacher classroom or lecture hall only 
in that the teacher is not physically present. 

Interactive multimedia wide area network 

Computer-based multimedia instruction, 
however, is an alternative to the conventional 
conception of television teaching. This 
alternative employs computer-based 
instruction as the centerpiece for an 



instructional system which extends the 
"electronic classroom" to distance education. 
The local area instructional network Is 
expanded to form a wide area network. As in a 
local area network, the wide area network will 
also include computers, CD-ROM and other 
types of optical data storage, video tape 
recorders, video disc players, and facsimile 
transceivers. 

The technology is now available for 
transmitting high quality pictures, voice, and 
data over wide area network. "T-3" 
telecommunications service, with a bandwidth 
of 54Mbps, using fiber optics cable is the 
transmission medium of choice. While this 
bandwidth is technically below the level 
required for complete "full-motion" video, the 
service does provide video transmission 
quality that appears to the viewer as 
indistinguishable from full-motion. Coder- 
decoder devices ("codecs") select, compress, 
transmit, and then expand that portion of the 
video signal essential to realistic viewing. 
Data and voice transmission are essentially 
unaffected. Combining microwave 
transmission for audio anrt video signals with 
high quality telephone lines for data 
transmission is another possibility. 

Since the "electronic classrooms" in a wide 
area network can function as workrooms or 
laboratories as well as a standard classrooms, 
the conventional presentation-recitation 
methods of teaching is inadequate to take full 
advantage of the available resources. Unlike 
conventional television teaching where the 
teacher often acts as resident expert and TV 
personality, the teacher In a multimedia wide 
area network needs to serve as model, manager, 
and mentor. Instructional designs will need to 
reflect the same type of changes required in the 
local area multimedia network. 

Problems and Resolutions 

Interactive instructional network development 
is not without problems, but we believe the 
problems are amenable to resolution. 

Technical 

The most serious technical problems we have 
found are those related to improvements in 
network and instructional resources. 
Hardware, especially that related to CD-ROM 
capabilities, is changing rapidly. With the 
advent of rapid search VCRs, video tape is 
becoming a more attractive interactive 
medium. Each software upgrade usually 
includes more features, uses more disk space, 
and often makes subtle undocumented changes 
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In operation. 

These Improvements, especially the latter type, 
may impede the work of the network. Earlier 
versions of Grolier's CD-ROM encyclopedia, for 
example, could be integrated into the network 
with relatively little difficulty; the latest 
version, with very useful graphics, has been a 
challenge. Upgrades in DESQview and 
LANtastic have a similar history. But 
overcoming these technical difficulties does 
have a marked positive effect on the Improved 
performance of the network. 

Cost 

The first questions usually raised about 
establishing a local or wide area multi-media 
instructional network are related to cost. 
Those costs, however, are a function of the 
hardware and software already available. 
Since there is no requirement for dedicated 
computers or video equipment, existing 
hardware can be adapted for network use. If 
purchased new, the hardware for a principal 
workstation/file server can be obtained for no 
more than $10,000; a remote station can be 
outfitted for under $1200. The Lantastic 
Starter Kit (including cards, cable, and 
software) is priced at about $400 for two 
stations with $225 for each additional station). 
DESQview with memory manager and utilities 
is priced at about $230. 

The cost of a wide area network is obviously 
much greater. These costs will vary according 
to distance, financing arrangements, and other 
local factors, but cooperation among 
educational institutions, telecommunications 
companies, and electronic suppliers can now 
result in a more favorable cost-benefit 
analysis. Two years ago, costs appeared to be 
prohibitive, but that situation is rapidly 
changing. 

In at least one instance, the yearly cost to a five 
school consortium for leasing the facilities 
needed for ' classroom door" to "classroom 
door" data, video, and television transmission 
has been quoted as about the same as the cost of 
employing a first-year teacher and less than 
purchasing a school bus. Television equipment 
is not Inexpensive, but many price levels are 
available. 

Teachers and Materials 

With the rapidly increasing Merest in 
multimediated instruction, cor ^uter, video, 
and CD-ROM materials are becoming more 
widely available. To be effective, however, 
instructional materials must be adapted to the 



educational context in which they will be used, 
and accepted by the teachers who will use them. 
As one part of a resolution to this problem, we 
offer at Illinois State an extended summer 
workshop on the development of interactive 
multimedia materials. In this workshop the 
participants are able to make a start in 
learning how to adapt existing instructional 
materials to the interactive network format, 
should they choose to do so. 

A more serious long-term problem is teacher 
preparation. The potential of multimedia 
interactive television can not be realized 
unless teachers have the knowledge, skills, and 
materials to make use of the technology. The 
knowledge and skills required for teaching 
with a multimedia interactive local or wide 
area network setting are beyond those which 
classroom teachers or university instructors 
can now reasonably be expected to have. Many 
of our colleagues in the schools and higher 
education are still struggling with 
Incorporating computers into their working 
lives, to say nothing of CD-ROMs and video 
discs. This is the classic problem of 
technological capability exceeding human 
capability to employ it. We have no ready 
solution, but we continue to raise the issue with 
teacher educators. 

Summary 

In summary, we believe a computer-based 
instructional multimedia network has 
significant educational advantages for 
teachers and students. When integrated with 
two-way television, such a network can also 
provide a truly proactive instructional system 
for distance learning. Both local and wide area 
networks can increase the productivity and 
reduce the cost of interactive multimedia 
instruction, thereby increasing its potential to 
positively effect educational change. Teachers 
must be prepared to Implement these networks, 
however, before such change can be realized. 
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ABSTRACT 

This paper details an ongoing research plan 
MICAL for the effective design and 
implementation of multimedia interactive 
computer aided learning (MICAL) in computer 
science. MICAL includes four progressive 
stages; Phase 1 : Utilization of barcode 
controlled video; Phase II : Development of 
selected multimedia workstations. Phase III : 
Incorporation of expert system interfaces to 
the Level III applications, and Phase IV : 
Transfer of these applications to a 
microcomputer-based local area network. 

This plan is tailored to enhance the 
instructional efficacy of the Department of 
Computer Science and the quality o. r 
information delivery systems at Western 
Kentucky University. 

OVERVIEW 

The advent of multimedia technology is 
changing the design and implementation of 
instructional delivery systems in both the 
corporate and academic environments. Using 
only a piece of chalk and a chalkboard has 
been equated to using "one stone age implement 
on top of another stone age implement " 
(Sneiderman, 1990). The proper blend of 
graphics, video, audio, and computer 
technologies has created a new "Virtual 
Reality" and "Visualization" for learning 
environments. Producing prototypes of a single 
user multimedia system is a step in the right 
direction, but it does not meet the needs of 
university lecture classes of forty or more 
students, or the needs of participants who 
require more time to master conceptual 
learning. The purpose of the MICAL research 
plan is to refine, develop, and disseminate 
effective video-based multimedia systems for a 
wide range of students in a computer science 
department. The target audience at Western 
Kentucky University ranges from advanced 
graduate students in computer science to 
multiple sections of first time computer users. 
The delivery systems will be utilized in the 
regular classroom, the department 
laboratories, and eventually the university 
laboratories. Common software and hardware 
platform(s) will be effected so that 



software /hardware implementations can be 
utilized, not Just by the Computer Science 
department, but the entire university campus. 

The main body of this paper discusses the four 
planned phases for the MICAL endeavor. The 
sections are as follows: 

Phase 1: Portable Barcode Controlled 
Videodisk Systems 

Phase II : Level III Videodisk Stations 

Phase III: Expert System Interfaces for 
Workstations 

Phase IV : Local Area NetwoiKS for Video- 
based 

Workstations 

The design constraints, environmental factors, 
and research questions for each phase will be 
highlighted in the rest of the paper. 

PHASE I: Portable Barcode Controlled 
Videodisk 8ystems 

The initial start of MICAL is to select those 
existing videodisks most appropriate to 
computer science instruction and to 
incorporate their utilization in the standard 
lecture class of twenty to fifty students. The 
first phase does not include in- house 
production of videodisks (even though this is 
possible) due to the time, cost, and man hours 
of such an endeavor. The target audience for the 
initial stage is not undergraduate or graduates 
in computer science, but the literacy students 
in an introductory course in computer 
technology. The reason for this selection is the 
expanding enrollment in this required course 
(CS145). Currently there are multiple sections 
of thirty-five students per semester who learn 
an introduction to microcomputer packages, 
programming, and social issues of technology. 
With limited resources and facilities, this 
heterogeneous lecture-oriented environment is 
the initial target audience. 

The system selected for this scenario is a 
videodisk player with barcode controls for the 
instructor. The videodisk player (with Level III 
capability) is managed by scanning barcode 
symbols from a textual page to control the 
videodisc. Hence, the instructor is in complete 
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control of the sequencing of the presentation. 
The system with Its attached color monitor Is 
mounted on a portable cart to permit easy 
portage to different classrooms. 

The videodlsks utilized are those consistent 
with the course syllabus: history of computers, 
introduction to telecommunications, hardware 
design, human factors, and applications. 
Videodlsks on microcomputer packages, other 
than DOS, are not utilized because of the 
potential change In versions of these software 
packages. The barcode development process 
includes searching for frame numbers of video 
sequences with a hand control, printing 
barcode representation of videodisk 
instructions and frame numbers, and merging 
the barcode symbols with appropriate textual 
instructions using a wordprocessor. 

The lesson is created in consultation with the 
Instructor, and refinements, changes, and 
resequencing are easily handled with a 
wordprocessor and barcode software. 

Research questions include: Which video topics 
are most successful in this selected 
environment?; Why?; What human interface 
factors afreet the instructor's utilization of this 
media system?; Can the software development 
cycle for this technique be easily transported to 
the Instructor?; and Is there any correlation 
between the video topic and the learning style 
of the student? Finding answers to these 
questions is the main goal of Phase I along 
with the acceptance of the videodisk as another 
teaching tool for the professor. 



phase n: 

Once videodlsks have been utilized and 
integrated into selected classrooms, the most 
effective ones will be ported to Level III 
workstations. The computer control will be 
either a Zenith 80336 and/or a Macintosh IIS: 
This is consistent with the supported hardware 
platforms in the Computer Science department 
and consistent with the Zenith platform in 
campus-wide laboratories at Western Kentucky 
University. The software utilized will be 
HyperCard-like packages which include 
Windows 3.0, Toolbook, and/or Authorware. 
Even though the initial development for Level 
III Videodisk stations will follow a Zenith 
platform, the Macintosh will be incorporated. 

Since the Computer Science department has a 
Zenith/led display system in each classroom, a 
videodisk under computer control Is viable. 
Both barcode and microcomputer controlled 



systems will be placed in the laboratories to 
supplement classroom Instruction and 
promote individualized learning. Videodisk 
topics will e pand to undergraduate and/or 
graduate components. Programming, software 
engineering, JCL, uatabase, architecture, and 
similar topics will be included. Since many 
computer related topics are now being stored on 
compact disc read only memory (CD-ROM), this 
technology will be merged into the multi-media 
stand alone stations. Research directions will 
include software development for the 
multimedia components, effective graphical 
interfaces, appropriate help/testing 
components, and possible correlations between 
learning outcomes and participant learning 
styles. The effective incorporation of the 
microcomputer with video, overlay graphics, 
and CD-ROM will be the primaiy objective of 
Phase II (Pigford. 85).(Pigford t 84). 

PHASE m: 

Phase II development of MICAL will be 
extended to expert system interfaces or expert 
front ends to the multimedia video 
workstations. An expert system is a software 
program that emulates the intelligence of a 
human expert in a narrow domain of expertise. 
An expert system will be used to control the 
interface to the video station, to diagnose 
students errors, to control sequencing in 
instruction and, in general, emulate the 
behavior of a master teacher with students in 
computer science. Video topics used for expert 
system development will be drawn from those 
topics which have proven to be most beneficial 
and efficacious In Phase I and II. 

The software tools for expert system 
development must be Incorporated with the 
authoring tools for the video workstation. 
Possible software solutions include VP-Expert, 
ISt Class. EXSYS, and Guru. VP-Expert is a 
viable candidate because of its integrability 
with existing software libraries (Pigford, 1990). 
Adding expert system interfaces and 
components will help to increase the efficiency 
and utility of the learning workstation, but the 
domain (video content) will tend to be narrower 
In scope (Baur, 1988). 

The relevant research issues in this phase are 
designing intelligent user interfaces, 
establishing relevant knowledge bases for 
selected domains, testing the expert systems, 
and integrating the various software 
components of the multimedia workstation 
(Devolder, 1989), (Ting, 1988). Transforming 
the video workstation into an Intelligent 
tutoring system Is the central goal of Phase III. 
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PHASE IV: LOCAL AREA NETWORKS FOR 
VIDEO WORKSTATIONS 
Phase IV of the research plan involves the 
expansion of the delivery system from 
essentially a single-user mode to a multiple- 
user mode. This extension will be 
accomplished by the use of multiple 
workstations connected with a local area 
network. The extension creates new problems. 
If the multimedia sources Include a vldeodisk. 
then only a single user can use the material at 
one time. Hence, the concept of multiple users 
cannot be Implemented. 

Second, if the multimedia Source includes a 
CD-ROM, there are severe limitations. A CD- 
ROM System is not capable of reproducing full- 
motion video and has a limited capacity 
relative to the storage of still frames. 

These limitations are caused by the fact that 
the storage of digital images (voice) takes a 
great deal of memory which is not the case with 
video stored on a video disk which is in analog 
format. The amount of storage needed for full- 
motion video sequences is so great that it is 
currently not possible, even at high transfer 
rates, to move the images fast enough across a 
network to be able to reproduce them at a 
playback rate of 30 images per second, a rate 
that is required for full-motion video. 

The purpose of the phase is to study and 
implement techniques that will permit the use 
of a local area network. A promising solution 
to the aforementioned problems is digital video 
interactive (DVI) which is under development, 
but the solution is still some time away. 

Successful completion of this phase will enable 
students enrolled In Computer Science classes 
at Western Kentucky University to use the 
systems developed in phases II and III of the 
project in a networked environment that will 
be cost effective (Lidard, 1990). 

SUMMARY 

This paper outlines a four phase research plan 
MICAL for developing multimedia workstation 
for the Department of Computer Science at 
Western Kentucky University. The goal of the 
research effort is to improve the quality of 
instruction for different levels of students In 
computer science. The software and hardware 
platforms are designed to be ported to the 
university environment. The four phases 
Involve the use of portable barcode vldeodisk 
configurations, Level III development, expert 
system interfaces, and transfer to local area 



networks. Constraints, environmental factors, 
and research questions for each of the four 
phases have been discussed. The research plan 
follows the usual "re-evaluate and change" 
spiral found in the traditional software 
development process. A time frame of three 
years is expected, depending on the 
hardware /software developments for a 
networked Implementation of the system. 
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AUTHORING SOFTWARE SELECTION 



Michael Collins, Memorial University of Newfoundland 



Abstract 

A comparison of the different types of 
authoring software included analysis of costs 
of use, programming abilities, ease of use, 
quality of resulting courseware, and usefulness 
across a wide variety of subject disciplines. 
Determining the actual costs of using 
authoring software was exceptionally 
difficulty due to the wide variety of fees 
associated with use of this software. TenCORE 
was rated best in terms of use, programming 
abilities, and courseware quality. PCD3 and 
ACT-III were the most user friendly. PC/PILOT 
was the least experience to operate. 

Introduction 

I have been Involved in courseware production 
on Apple II's and Commodore 64*s since 1982, 
but it is only since 1988 that I have been 
Involved In the authoring of courseware for 
MS-DOS machines when I received a grant to 
evaluate a number of different types of 
authoring software. 



Purpose 

The primary purpose of the project was to 
evaluate a number of authoring languages and 
authoring systems to determine which were the 
best for use in authoring courseware in a 
variety of subject areas at the post-secondary 
level. 

Evaluation 

The evaluation was to take account of a number 
of factors including, 

a. cost (including purchase price, annual 
licensing fee, maintenance fee, etc.) 

b. hardware requirements (for both authoring 
and student use) 

c. ease of use and operation 

d. usefulness across a wide variety of subject 
disciplines 

e. quality of resulting courseware. 

Criteria for selection of authoring software 

These are the criteria used for the initial 
selection of authoring software. 

1. must run on IBM-compatible computers 

2. must require no more than 512K RAM, and 
use either one or two 5.25" 360K floppy 
drives, and/or hard disk, or any 
combination 

3. must incorporate both colour and graphics 



4. must use the EGA graphics standard 

5. should not cost more than $2500 
(Canadian) 

6. should be readily obtainable. 



Identification of authoring software 

The first step was to review the research 
literature to identify authoring software and 
any available reviews. Our search indicated 
that there was no single list of all the 
commercially available authoring software, 
and so we had to rely on a number of sources 
(Barker and Singh, 1982; Kearsley, 1982; 
Burger, 1986; Barker, 1987; Tyre 1988, 1989). 
Even a more recent list (Greenfield, 1990) lists 
only 34 and this omits a number of quite well 
known and still commercially available 
authoring software. 



Reviews of authoring software 

The literature Includes many articles which 
review authoring software, but often these are 
technical reviews focusing on programming 
abilities with no attempt to produce 
courseware, or use it with the Intended target 
audience, the Instructors. Very rarely Is there a 
mention of the actual cost of using courseware 
generated with the software. Some authors 
have reviewed software as regards its potential 
usefulness to a particular subject discipline. 
Two types of authoring software which were 
often mentioned in the literature were 
PC/PILOT and TenCORE, with the latter 
receiving the better reviews. Identifying 
potential authoring software, then, is still very 
much a shot in the dark approach. 



Method 

Since 5 different aspects of authoring software 
were to be evaluated, five different methods 
were used, one for each aspect. 

Cost 

This information was obtained directly from 
the manufacturers of the software. 

Hardware 

This information was obtained from the 
manufacturers or their publications. 
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Ease of use/programming abilities 

Initially it was planned to evaluate seven 
different types of software, and, with all the 
work involved, it was decided to hire students 
to learn how to use the software, and to create 
example courseware. Each student was first to 
learn to use PC/PILOT, the language with which 
I was most familiar, and then to learn the use 
of two further tools. In this way we could 
compare the software using PC /PILOT as the 
common factor. 

Usefulness across a wide variety of subject 
disciplines 

Quality of resulting courseware 

A number of instructors agreed to develop short 
lessons for programming with each type of 
authoring software. Each was asked to develop 
a sample lesson using as many different 
programming techniques as they thought 
would be routinely used in their subject area 
e.g. colour, graphics, animation, super- and 
sub-scripts and the like. Once the courseware 
had been developed each author would then be 
asked to evaluate the usefulness of each 
programmed version to that subject area, and 
to rate its quality. 



Results 

The project was originally designed to run 4 
months, but within the first few weeks a 
number of major problems were encountered 
which caused the project period to be extended. 
Some of these problems are listed in Table 1 
and are referred to as implementation 
problems. As a result we returned one tool, 
MICROINSTRUCTOR, for a full refund, and in 
its place ordered a further three products. 

All the authoring software eventually used in 
the project is listed here in Table 2. 

Hardware requirements 

Even though the hardware requirements had 
been checked with the manufacturer prior to 
ordering the software a number of problems 
were encountered with RAM requirements, type 
of diskette on which the software was supplied 
(720K 3.5",360K 5.25", 1.2M 5.25"), and 
peripherals (graphics tablet etc.). 



SOFTWARE 

AUTOOL* 



CSR Trainer 

Microinstructor 

PCD3 



PC/PILOT 

TenCORE 

VITAL 



PROBLEMS 

required additional hardware/peripherals. (Matrox card; multisync 
monitor; 640K RAM) y 

set up problems - proper procedures missing from manual; company had 
to be contacted for correct procedures. 

diskettes would not run on IBM-compatibles. Software returned for full 
refund. 

software took almost 3 months to arrive, and lacked necessary set-up 
disks when it did. Obtaining this disk required phone calls to 3 different 
North American cities and the disks were not obtained for another 
month. 

quick delivery and no problems, 
prompt delivery and no problems. 

lequired additional hardware - a second monochrome monitor in 
addition to the EGA monitor. Manual and diskette versions did not 
coincide and both had to be replaced. Graphics tablet driver didn't work - 
multiple replacements also didn't work. 



Table 1 Problems Encountered in implementing the software 
• now called HyperTRAINTable 2 
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Software evaluated: 

•ACT-ffl (Informatics Group, Inc.. West Hartford, Connecticut) 

•CSR-Trainer 4000 (Computer Systems Research Inc., Avon, Connecticut) 

•GUIDE (Owl International Inc., Bellevue, Washington) 

•HyperTRAIN (Hofbauer Information Systems, Dallas, Texas) 

•Mlcrolnstructor (C.V. Mosby Ltd., St. Louis, Missouri) 

•Micro NATAL (Softwords, Victoria, British Columbia) 

•PCD3 (Control Data Corporation, Minneapolis, Minnesota) 

•PC/P T LOT (Washington Computer Services, Bellingham, Washington) 

•Smalltalk/V (Digitalk Inc., Los Angeles, California) 

•TenCORE (Computer Teaching Corporation, Champaign, Illinois) 

•VITAL (Com:port International, Ottawa, Ontario) 

Table 2 



Student Programmers Rating of Authoring Software 



Software 



ACT-III 4.0 

CSR Trainer 3.5 

GUIDE 0.0 

HyperTRAIN 30 

Mlcrolnstructor n/a 

Micro NATAL 2.0 

PCD3 9.0 

PC/PILOT 6.0 

Smalltalk/V 2.0 

TenCORE 10.0 

VITAL 2.0 



Would be 7 or more if included ega graphics, looping and firrays 



For experienced programmers would be 5.5 



For experienced programmers would be 8.0 



Table 3 

Ease of use/programming abilities 

The original plan of students evaluating 2 types 
of software in addition to PC/PILOT became 
impossible to continue with because of the 
problems which were encountered. It was 
decided, therefore, to create a series of stages 
through which the software had to pass. If 
software did not successfully pass a stage, then 
its use in the project was terminated at that 
point. These stages were: 

1. setting up 

2. learning how to use software 

3. determining programming abilities 

4. programming example courseware 

Once students had learned to use the software 
they were then asked to rate it. In determining 
this rating the students took account of the 
purpose of the project which was to identify 
software which cculd be used by novice 
computer users. The ratings are shown in 
Table 3. Clearly TenCORE and PCD3 had the 
highest ratings. It should be noted that this 



evaluation was conducted without knowledge 
of the Individual programming requirements 
of the sample lessons. 

Usefulness across a wide variety of subject 
disciplines and Courseware quality 

Only four of the different types of authoring 
software were successfully used to programme 
all six of the sample lessons. Of these only 
PC/PILOT and TenCORE produced all 6 sample 
lessons. PCD3 could only be used to 
programme courseware In Biology, Chemistry 
and Psychology, and ACT-III only In Biology. 

Each author was then asked to review the 
resulting courseware and state whether each 
version was acceptable, and if more than one 
was acceptable, to rate them in order of 
preference. The results are shown In Table 4. 
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Author Rating of Courseware Quality 



Subject Area 



Software 



PCD3 



PC/PILOT 



Biology 

Chemistry 

English 

Mathematics 

Physics 

Psychology 

1 = First; 

Table 4 



*3 

*3 



*3 

2 = Second; 



•1 
*2 
*2 
*2 
*2 
*2 

3 = Third; 



TenCQRE 



All the created courseware was of an acceptable 
standard. In all cases but one TenCORE was 
rated as best, with PC/PILOT second and PCD3 
third. However authors felt that PC/PILOT's 
standard font was easier to read than 
TenCORE's. while PC/PILOT could not display 
custom characters In EGA mode. Each author 
commented on PCD3's long loading time and 
Its requirement for two floppy drives or a hard 
drive. 

Cost 

Estimating the cost of using authoring software 
would seem to be a straightforward task, but, 
unfortunately, this turned out to be one of the 
most difficult aspects of the evaluation. The 
advertised purchase price of an authoring tool 
Is rarefy, if ever, the actual cost of using that 
tool, since many firms charge a variety of 
different fees for the authoring of courseware, 
and actually using it with students. The 
various costs which may be associated with the 
use of authoring software are as follows. 

1. authoring fee 

2. authoring options 

3. additional hardware/peripherals 

4. student use fee 

5. distribution fee 

6. commercial distribution license 

7. annual maintenance fee 

8. training fee 

The authoring fee is the one most often 
advertised as the price of authoring software. 
Often, however, this fee covers only the use of 
the software on one single authoring machine. 
Multiple authors on the same campus would 
usually need to purchase a site license or obtain 
multiple copies of the software. 

Some of the software also offer options or 
enhancements which are additional to the base 
price. PC/PILOT, for example, offers an 
Advanced Features Library (AFL), and 



•2 
•1 
•1 
•1 
•1 
•1 



* DM 



= Acceptable 



TenCORE offered TenCORE Assistant, an 
authoring system overlay for TenCORE. 
Several also offer the option of using a third 
party graphics package such as PC Paintbrush. 

Some may require the purchase of additional 
hardware or peripherals, and even though 
these are not strictly speaking authoring 
software requirements, account should be 
taken of them in calculating the cost of use. 

A number of companies charge a separate fee 
for student use of courseware created with their 
authoring software. CSR Trainer 4000, for 
example, charged $95 per workstation. Yet 
others charge for the use of this courseware off- 
site. TenCORE, for example, charged a 
perpetual fee of $1200 per item of courseware 
used off-site. There may also be fees for 
commercial distribution of courseware created 
with authoring software. 

Some companies charge an annual 
maintenance fee for the use of their products. 
This fee may be a fixed amount or a percentage 
of the total fees for the software. The 
maintenance fee usually allows the purchaser 
to obtain free software and documentation 
updates, and phone support. 

A number of companies offer training sessions 
in the use of their products. TenCORE offers 
both on- and off-site expert training in the use 
of their products. 

In or hr to estimate the possible costs of using 
authoring software it was decided to cost out 
the use of authoring software under a number 
of different scenarios as shown here. 

Scenario #1 

One authoring machine producing courseware 
only to be run on 5 student machines on the 
same campus. 
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Scenario #2 

One authoring machine producing 10 Items of 
courseware to be run on 5 student machines on 
the same campus, and also to be used on (I.e. 
distributed to) 100 student machines at 10 
other sites. 

Scenario #3 

10 authoring machines producing 10 Items of 
courseware to be run on 50 student machines 
on thi same campus, and also to be used on (i.e. 



distributed to) 100 student machines a f 10 
other sites. 

The estimated costs for the leading four 
authoring tools are shown In Table 5. Among 
the four leading contenders It Is clear that 
PC/PILOT Is the least costly of the four for each 
scenario, followed by ACT-HI. TenCOREs price 
Increases enormously for mul.iple author use 
and off-site use. 



Table 5 

Estimated Costs of Using Authoring Software Under 3 Different Sets of Scenarios (All prices converted 
to Canadian dollars; US - 1.18C$) 



Software _ 


Scenario 1 


Scenario 2 


Scenario 3 


Notes 


ACT-III 


584 


4,673 


4,o7o 


v A iica or HletiHViiitlnn fpoc Mn 2>n m 
INO olUUCril USC Ul lllalllUUllUIl ICvdi i^u cUAiiuai 








maintPTir^iPP f*pp 

lllClllllCAlC> ICC ICC. 


AUTOOL 1 


3,108 


3,108 


19,328 


Prices Include student use fees. Annual 






maintenance fee of 12%. 


CSR Trainer 


2,304 


10,491 


26.122 


Students use fees on and off-site. Annual 






maintenance fee of 15%. 


GUIDE 


673 


>673 2 


>2,891 2 


Distribution fees and student use fees not specified. 


Micro NATAL 


1,495 


6,495 3 


9.995 


Fee for Interpreters on off-campus machines. 








Annual maintenance fee of 15%. 


PCD3 6 


14,535 


14,535 


22,500 


No student use, or distribution fees. No annual 




maintenance fee. Authoring costs Include 
training, hot-line support, software and 
documentation updates. 


PC/PILOT 


189 


189 


1.888 


No student use or distribution fees. 


(->873) 4 


(-> 873) 4 


>649 5 


No annual maintenance fee. 


Smalltalk/V 


>83 


>83 5 


Distribution fees, not yet set. No annual 








maintenance fee. 


TenCORE 


2,938 


17,771 


31.825 


Distribution fee of $1200 per product (one time 






only fee). Maintenance fee of $570. 


VITAL 


2,220 


=2,820 


=22,800 


No student use fees. Distribution fee of about $5 per 








disk. Maintenance fee of 20%. 



now caller HyperTRAIN 

these are minimal since the applicable fees were not specified. 
Interpreter fee varies from $50 to $150 each, 
higher price Includes authoring aids which are deemed essential, 
appropriate fees not yet fixed by manufacturer. 

when PCD3 was originally purchased It cost $300 (US). As a result of changes within 



Control 



Table 5 
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Conclusions 

The original purpose of the project was to 
Identify authoring software which could be 
used by Individual instructors in a variety of 
subject areas at the post-secondary level. We 
have concluded that unfortunately it is not 
advisable for a novice computer user to use MS- 
DOS authoring software on his/her own. The 
potential author should have a working 
knowledge of an IBM-type computer, and be 
familiar with MS-DOS operations as well as a 
programming language such as BASIC. The 
potential author may also need technical 
expertise to set up some of the software and to 
attach peripherals. The author also needs to be 
acquainted with the types of problems which 
may be encountered in identifying, ordering 
and implementing authoring software. 

Ideally, therefore, authoring should be a co- 
operative effort involving subject experts, 
programmers, instructional designers, and 
graphic artists. Unfortunately, however, in 
many institutions authoring will remain the 
preserve of interested instructors, who will 
have to produce courseware without much in 
the way of back-up facilities and support. 

As regards the actual software evaluated 
TenCORE was rated the best in terms of use, 
programming abilities and quality of 
courseware produced, and together with 
PC /PILOT, was successfully used to produce all 
6 sample lessons. PC/PILOT, however, was 
much less expensive to operate under a number 
of different scenarios. No start up problems 
were experienced with either of these. PCD3 
and ACT-HI were rated as very user friendly 
and easier for a novice to use than either 
PC/PILOT or TenCORE. Both PC/PILOT and 
TenCORE are well established authoring 
software and are frequently mentioned in the 
research literature. 

It should be noted that the shortcomings of 
much of the authoring software can be 
overcome by the use of external subroutines 
written in programming languages such as 
BASIC and PASCAL. ACT-III, PC/PILOT and 
TenCORE, for example, all allow the use of such 
programming subroutines. 

Even though most authoring software provides 
special graphics editors, we found it more 
^seful to produce graphics with a third party 
graphics package such as PC/Paintbrush or 
Paintshow Plus. Both PC/PILOT and TenCORE 
are set up to allow the use of PC/Paintbrush, 
and ACT-III actually uses PC/ Paintbrush as its 
own graphics editor. The use of PC/Paintbrush 



also allows the transfer of graphics between the 
three types of software. A handscanner, such as 
Logitech's, can be used with such graphics 
packages to enter graphics more easily. 

At the beginning of the project we decided to 
evaluate the usefulness of the authoring 
software for a number of subject areas by seeing 
if each type of software could be successfully 
used to programme all the requirements of six 
different subject area sample lessons. At the 
end of the project it became apparent that even 
courseware in the same subject area may 
require the use of quite different programming 
techniques. Rather than deciding which 
software should be used to programme a 
particular subject lesson on the basis of subject 
area alone, the courseware developer should 
identify the programming techniques 
necessary for the work, and then identify the 
authoring software which can handle all these 
requirements. Some of the more important 
programming/ authoring requirements 
identified in this study are shown in Table 6. 

It should be noted that new authoring software 
is continually coming onto the market (e.g. cT, 
Linkway) while established software is being 
upgraded. It should also be noted that there are 
now authoring overlay systems for both 
PC/PILOT and TenCORE, namely PROPI and 
TenCORE Producer, and both seem to be highly 
rated in the literature (e.g. Fisher, 1989). 

Footnote 

As a result of our project findings we have 
obtained a site license for PC/PILOT and now 
use it as our principal authoring software. We 
distribute copies to interested instructors, and 
also offer a limited authoring service using 
Computer Science majors, who do the 
programming under the guidance of myself and 
an assistant. We also have ACT-III and cT 
available for use by instructors although 
neither are supported by us at present. We have 
also bought a BBC-BASIC converter used by a 
chemist to bring BBC courseware Into the MS- 
DOS environment. 

We use PC/Paintbrush 4.0 as our main graphics 
editor and also use a Logitech handscanner. 
PCX Is also available to enter graphics into 
normally non-graphic oriented programming 
languages. At present we have developed some 
16 Biology programmes and one each in 
English and Mathematics. PC/PILOT is also 
being used to develop computer/ videodisc 
programmes in Physics. 
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Authoring/Programming Requirements 



1. Answer handling routines (multiple choice, answer precision) 

2. Ability to accept long answer strings 

3. Handling mathematical calculations 

4. Alternate character sets 

5. Proper super/subscripts 

6. Advanced programming abilities (dimensioning, variables, loops, for/next subroutines) 

7. Graphic/text split screens 

8. Colour (CGA, EGA, VGA) 

9. Complex graphics (i.e., free-hand drawing) 

10. Animation 

11. 3 dimensional objects 

1 2. Number of simultaneous colours (4, 1 6, 32, etc.) 

13. Alternative text fonts 

14. Rotation of objects 

1 5. Use of external graphics packages 

16. Ability to scan in text, graphics 

17. Ability to move out of authoring software into another programming environment (e.g., Pascal, CJ 
ia Ability to print out material for students 

19. Ability to interface with video 

20. Ability for student to edit text on screen 

21. CML option 



Table 6 
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DEVELOPING CBT COURSES FOR END-USER TRAINING 



Douglas MacKenzte, DMC Ltd 



Abstract 

Training end-users of computer systems has 
always been a major application of CBT. 
Nevertheless, creating CBT in parallel with 
system development or for systems subject to 
frequent change has proved difficult. This is 
due in part to current development methods, 
both the programming tools used and the 
design techniques governing them. CIRCUS is a 
mix of design philosophy and programming 
toolkit, building on prototyping techniques, 
which provides an environment for the 
creation of flexible, easily- updated courseware 
with substantial time and cost savings. In 
addition to proving its worth in its original 
application area of end-user training, CIRCUS 
has turned out to be an effective method of 
developing multi-lingual courseware and 
portable courses for different hardware 
configurations 

Training end-users of Information Technology 
(IT) systems was one of the earliest, and 
remains one of the most fruitful, application 
areas for computer-based training (CBT). In 
training terms the medium's suitability is 
obvious: a simulation of the system can be 
offered to the trainee in which carefully 
structured experimentation may be carried out 
in an environment where mistakes can safely 
be made and corrected and where real data are 
not at risk. 

The commercial benefits are equally clear. An 
organisation introducing new computer 
systems has a large training need thrust upon 
it. Many people need to be trained in a short 
period of time and these staff are likely to be 
scattered over several different sites in 
different regions, perhaps even in different 
countries. The cost of taking trainers to remote 
sites, or bringing trainees to central 
classrooms, makes distance learning a 
worthwhile option. In some instances it may be 
possible to use the real computer system for 
training but often this is not so. For example, 
use of live data may not be practicable or users 
may need to be trained ahead of system 
completion so that they will be proficient and 
productive from day one. Even where training 
databases are proposed as part of the system, 
experience shows that when development 
schedules begin to strain, the training 



component is usually the first victim and the 
period between the projected availability of the 
system for training purposes and the "go-live" 
date becomes ever shorter. This is only the 
practical problem. There remains the question 
of whether it is wise to offer the trainee a full 
simulation of a real system for training. The 
research into high-fidelity simulation for 
novice users (e.g. Alessi, 1988) may be 
summarised as saying that an accurate 
representation allows the trainee to become as 
lost and bewildered as would be the case in real- 
life. However, despite CBT's inherent 
suitability for end-user training there are 
significant problems in Its implementation 
which have discouraged even greater use. 

If a CBT course is to be ready ahead of the 
arrival of a new computer system or a major 
upgrade, the course's design must be based on 
the "real" system specification. Unfortunately, 
these fixed specifications have a habit of 
changing dramatically during the life of a 
project. When this happens, the simulation of 
the system in a CBT package has to change also: 
a process which is both time -consuming and 
expensive. Until now that has simply been 
accepted and the received wisdom has been not 
to tiy to develop CBT for unstable systems. 

There are cases, though, when the problem 
cannot be side-stepped. There may not be time 
to develop a CBT course between the date the 
computer system becomes stable and its 
implementation. On other occasions there may 
still be a huge training need before the final 
release of the software. In these cases there is 
no choice but to develop a CBT course in 
parallel with the creation of the real system. 

Probably because commissioners of CBT have 
stuck to the 'don't develop CBT for an unstable 
system* maxim, little thought has gone in to 
how CBT software should be designed for 
situations where changing system 
specifications must to be tracked during course 
development. Yet the notion underlying the 
demand for stability is absurd* Why should the 
resources required to modify a superficial 
simulation of a system be anything like as 
great as those required to change fundamental 
system elements such as database structures, 
networking protocols or access methods? 
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Even when the computer system for which 
training is to be provided is stable, the time 
required to create CBT courses by traditional 
methods is often too long for a course to be 
available by the time the real ystem is 
implemented. Estimates of the ratio of hours of 
development time to course duration vary 
enormously from 70:1 to 1000:1 (Jay, Bernstein 
and Gunderson, 1987, Sampath and Quaine. 
1990, Friedler and Shabo, 1991) with a 
commonly quoted figure in the UK of 275: 1. 

For many commercial training applications 
this figure is simply too high. The most 
common approach to reducing development 
times is to search for an authoring system to 
simplify course creation* The costs in terms of 
lost flexibility incurred by using such a system 
are well known (Merrill, 1985), Although 
authoring systems whose primary purpose is 
the development of training for users of 
computer systems exist, a 'silver bullet 1 
approach is likely to be especially 
unsatisfactory in this application area. The 
simulation capabilities, let alone the 
instructional strategies, required for training 
senior managers how to use a graphically- 
based Decision Support System are very 
different to those required for instructing 
experienced data entry clerks in the operation 
of a mainframe-based customer service 
database. 

Another possible solution is using a self- 
developed code generator (Bryant, 1990) to 
automate routine features used throughout a 
particular project. This has its attractions but, 
when it becomes simply a series of templates 
for different screen types, it is nothing more 
than a user-designed authoring system. If it is 
to have a life beyond one project its design is 
likely to be as limiting as a commercially 
produced authoring system but, given the 
development effort, at a far higher cost. 

The problem is not simply one of the 
availability of suitable programming tools: it 
is the whole role of programming in the 
courseware development cycle. Most 
Instructional Systems Design (ISD) approaches 
emphasise instructional design at the expense 
of development. Indeed it seems CBT software 
development always lags several years behind 
general purpose computing. Advances in 
software engineering, knowledge 
representation or object-oriented 
programming have made little mark in CBT. A 
few worthwhile attempts to highlight the 
importance of the programming side of CBT 



have appeared (Janossy, 1986, Wey Chen and 
Shen, 1989,) but on the whole, CBT authoring 
tools seem to encourage unstructured program 
design and coding. (Think how many so-called 
authoring languages do not even support 
subroutines.) 

In CBT courses to train end users of computer 
systems there seem to be two extremes. In 
courses developed with simpler types of 
authoring package, system screens tend to be 
represented by static images with 
accompanying text and little or no simulation 
of the computer system. This is hugely 
unsatisfactory from the training point of view 
as it offers little more to the trainee than what 
Is available in a well written manual. 
Alternatively, in courses written in more 
complex authoring languages or general- 
purpose programming languages, the 
simulation of the system screens and the 
functions of the live system are not 
distinguishable in the code from the pedagogic 
element. This has no adverse affect on the 
quality of training but, if the real system is 
modified by the removal of one compulsory 
input field, the changes required to make the 
CBT reflect this can be complex and time 
consuming. 

A series of CBT development projects for 
different clients to train users in the operation 
of various custom-built computer applications 
highlighted these problems and provided the 
Impetus for CIRCUS. There was, and remains, a 
determination not to create a new authoring 
system or language whether as a fully-fledged 
product or as an In-house code generator. Staff 
already had much experience in developing 
successful CBT solutions for end-user training 
so the quality of training was already there: the 
concentration would be on reducing 
development times and allowing courses to be 
modified more quickly. This meant building 
on existing working practices, particularly the 
technique of Instructional prototyping, and re- 
evaluating the role of programming in 
courseware creation. It is this background, and 
the fact that it evolved through use In several 
different projects, which makes CIRCUS 
something of a hybrid: part design philosophy, 
part programming toolkit and part 'the way we 
dc things here 1 . As such, it is hoped that it is 
general enough for others involved in the 
development of CBT for end-user training, to 
adapt to their own methods of working and 
their own preferred authoring or programming 
languages. 
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CmCUS and Its place in Prototyping 

Abandoning the ISD method of development 
can reduce course development time and 
instructional prototyping, a specialised 
application of incremental system 
development, (MacKenzie, 1990a) is one 
approach. The experiments with CIRCUS to 
date have been carried out with this method of 
development. The prototyping strategy deals 
with the entire design/development/ project 
management process: CIRCUS is concerned 
solely with the structure of the courseware 
code. CIRCUS could also be used with 
traditional ISD methods where the main 
objective is to provide a course which can be 
maintained easily after its first release. 
However, the benefits achieved in terms of 
faster user approval, more rigorous testing and, 
consequently, faster and cheaper development, 
which was the initial objective, have 
commended the prototyping route in projects 
carried out so far. 

The development to course duration ratio for 
IT-related courses developed using 
Instructional prototyping lies between 70 and 
160:1 (MacKenzie, 1990b). Significant 
development time improvements for the 
creation of most courseware by this method 
accrue for a number of reasons (MacKenzie. 
1987). The most important benefit in using 
instructional prototyping for courses 
Involving simulation of IT systems is the 
constant availability of prototypes for 
formative evaluation throughout the project. 
Subject-matter experts unlikely to be able to 
approve traditional paper-based course 
designs, either because of demands on their 
time or because of the design's inherent 
Incomprehensibility, have working courses the 
accuracy of which they may verify quickly and 
easily. This allows changes to be made as early 
as possible in development when they are 
considerably cheaper (for an analysis of the 
cost differences in software changes at 
different project stages, see Boehm, 1976). 

Prototypes are flexible. A key requirement is 
that prototypes be modified according to the 
comments of trainers, subject matter experts, 
instructional technologists, supervisors and 
potential trainees among the project 
participants. This may happen in a project no 
matter what method of working is adopted but 
CIRCUS provides an environment which 
affords cost-effective flexibility. A modular 
toolkit approach allows changes to the course, 
and in particular the system simulation 
elements, to be made simply. System changes 
are, therefore, quickly reflected in the CBT. 
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In addition to the rapid reflection of real 
system changes in the courseware, there is 
another benefit of prototype creation. The 
specification documents for the IT system will 
often represent several man-years effort but 
not only will there be ideas which seemed 
reasonable in the design document but look 
less good In a working model (Wasserman and 
Shewmake, 1985), but given an example of how 
a finished product will look, users' perceptions 
of requirements will frequently alter. A 
prototype CBT course concentrating on the 
simulation of system functions and the user 
interface may well be the first real product a 
user of a potential system sees. Changes in 
specification are far easier to spot at this stage 
than in the period of reviewing paper-based 
designs. Just as course changes are cheaper to 
make at this stage, system modifications are 
also considerably less expensive. Instructional 
prototypes in IT system training are not just a 
mean' of eliciting comment on the CBT course: 
they also offer a way of Influencing design and 
development of the IT system itself? 

The Principle of CIRCUS 

CIRCUS is a series of techniques which enables 
the prototypes described above to be created, 
and whenever necessary modified, quickly. As 
was pointed out in the introduction, CBT 
courseware involving simulation of IT systems 
generally does not separate the system element 
from the training element. This can make 
course modifications necessitated by changes 
in the computer system extremely 
cumbersome. As part of the preliminary 
fieldwork a course which had been written in 
an authoring language which did not allow the 
use of subroutines (Trainer 4000) was examined 
from the point of view of the modifications 
necessary were an extra input field to be added 
to the live system. In the relevant course 
section, about twenty minutes of study time, 
sixty eight changes had to be made to the code. 
It was estimated that if the course had been 
written in a conventional way in a more 
structured language such as PC/Pilot or 
TenCORE, around twenty to thirty code changes 
would still have been required. 

Where languages allow course creation with 
external editors, global change commands and 
internal macros may reduce the physical effort 
of making changes to the code. This is, 
however, likely to be complicated by embedded 
colour and highlighting commands or by the 
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Figure 1 The CIRCUS model divisions 



conjunction of tutorial text with system 
screens. In addition to the effort of changing 
the code in many places, a large demand is 
placed on project time by the new testing 
required. 

CIRCUS aims to reduce the number of changes 
in the CBT caused by a change to the real 
system. Hie basic id^a is to keep the system and 
the instructional elements separate with 
pointers from one to the other. The system 
information is divided into screen displays 
and field definitions and the training element 
constitutes a third section. The "connections" 
between the three elements are defined in the 
central control module. This format is 
illustrated in Figure 1. 

Screen displays 

The screen display portion of a course contains 
definitions of the system screens as they 
appear to a system user. These are passive 
overlays with no distinction between fields: 



input, output or mixed fields are simply left as 
blanks. 

Field definitions 

This section contains definitions of input 
fields for each screen: field name, screen co- 
ordinates, input field length, any type checking 
done within the field, the keys which allow 
movement to the next or previous field, the 
names of those fields. These definitions could 
be coded explicitly for each field. However, the 
first CIRCUS toolkit implemented in PC/Pilot 
uses a shorthand. So called operating variables 
are set in each field definition which are passed 
to Pilot subroutines. For example, on the first 
project where CIRCUS was used, the real 
computer system used 80x24 text screens with a 
mixture of fields. Filling an input field meant 
the cursor automatically Jumped to the next 
field. Attempting to input data in an output 
field caused an error tone to be sounded but 
required the TAB key to be pressed before the 
cursor moved on. The CBT version of this 
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involved setting variables mlth to 8, Jkey to 9 
and num to 1. The subroutines in the control 
module use these to determine that the 
maximum length of input is eight characters 
and so adjusts the input window accordingly. 
Onfy numeric characters are to be accepted and, 
once the field is full or the tab key (ASCII value 
9) has been pressed, the cursor will 
automatically Jump to the start of the next 
field. This, in CIRCUS terminology is called an 
operating definition. 

In addition to the detailed operating definition 
there is also a higher level field description, the 
navigational definition. This describes a 
particular screen in terms of a list of the fields 
used on a screen in the order they would be 
completed. Within this definition there is space 
for event markers. These are checks for author- 
specified conditions such as 'field traversed 
three times by trainee without inputting 
anything' or 'non-numeric input attempted' or 
a test for a specific input. The use of these is 
described in the Implementation section below. 

Training 

The training portion in CIRCUS contains the 
purely explanatory parts of a CBT course. These 
may be graphics, animation sequences or 
blocks of text. They may be individually 
labelled items, such as a textual explanation of 
how to complete a particular field, or quite long 
sequences explaining, say, the operation of 
TAB and BACKTAB keys, with branching 
instructions limited to that particular 
sequence. (It is very important for the modular 
nature of CIRCUS development and the ease of 
0 debugging and testing that the blocks of 
training material are self contained and do not 
contain references to other blocks.) 

Control Module 

This section is the driver for the v hole course. 
Screen overlays are called from here with 
pointers to the training elements to be used 
with them. The field definitions to be used with 
each screen are specified. If a section of a course 
Is to be a very close approximation to the real 
system, as in a mastery test, whole screen field 
definitions are called using the navigational 
definitions described above with the 
appropriate event markers primed to afford the 
specified help or remediation when triggered. 

A purely tutorial part of the course would 
probably use only an operating definition often 
with some of the operating variables turned off. 
This is called along with the appropriate 
teaching material portion. 
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The Implementation of CIRCUS 

CIRCUS was originally conceived simply as a 
development method and so capable of being 
used with a wide variety of languages. In-house 
use of it at DMC has, however, led to a large 
number of supporting routines being developed 
in PC/Pilot. 

There were two reasons for this choice. The 
power and flexibility of PC/Pilot make it a very 
suitable choice for creating complex 
simulations of IT systems and so it had already 
become something of a house standard for this 
type of course. Secondly, the language uses a so- 
called execute indirect statement which allows 
a string to be executed as a language command. 
For example, an event marker might be of the 
form adpas>3, a counter noting that a certain 
field had been passed more than three times, 
and a teaching segment marked with the label 
•INT1. These components can be combined in a 
string of the form rec$="u(adpas>3):INTl". 
Executing the rec$ string as a command means 
that when the condition in the event marker 
has been fulfilled, the teaching segment is 
called as a subroutine. This language feature, 
equivalent to having subroutines to which 
addresses, variables or conditions may be 
passed, is of immense value in adding 
automation to the toolkit. 

The Impact on Development Times 

The first project on which CIRCUS was used 
was a PC-based CBT course to train end users of 
an enquiry logging and tracking system used in 
a large UK bank. The logging system was 
developed in COBOL and ran on networked PCs. 
Trainees had neither knowledge of previous 
manual procedures used in the bank nor had 
they prior computing experience. There was a 
three phase implementation of the live system 
with significant changes with each release. 
CBT had to be available for users of each 
release. The live system was menu -driven with 
eight menu options. Complexity varied between 
four and thirty-two 

fields per screen, most of these both input and 
output fields. The average time to complete a 
transaction on the live system Is around five 
minutes. 

The first information available to the CBT 
developers was a draft functional specification 
of 6the real system. From this a prototype 
course was constructed with PC/Pilot showing 
how a system simulation would operate and 
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ninutee of trainee study developmentto activiiee 

time houmofoourse) 



Phase 1 


60 


140:1 


Bidding routines, screen 
displays and field defs. 


Phase II 


20 


90:1 


Changing navigational 
definitions, training 
sequences and event 
markers. 


Phase III 


30 


64:1 


Changing operating 
definitions. 
Constructing new 
exercises from existing 
oewwoons. 

Adding new training seqs. 



Table 1 Development Time Ratios for Courseware created unslng CIRCUS 



how data entry would be taught. This was 
achieved and made available to bank trainers 
and subject matter experts within five days. In 
addition to producing a design template on 
which non-CBT specialists could offer 
meaningful comments, the prototype 
Illustrated a number of inconsistencies within 
the functional specification itself and a 
number of features were clarified and user 
requests passed back to system developers. 

The CBT development continued in parallel 
with system developments which went through 
three pre-release software versions. A two-hour 
CBT course using a mixture of text-based 
system screens and EGA graphic Images, of 
which training on system use constituted 
around one hour's worth of material was ready 
at the time the system went live. The 
development time for this section was around 
140 man hours. This is fast compared to 
industry standards but was in line with the 
developers' expected ratios for such a course 
created using instructional prototyping and 
with a stable system specification (MacKenzie, 
1990b). Given the fact that the specification 
was not stable and the large number of changes 
to the system being simulated, CIRCUS 
certainly kept development time, and 
consequently cost, down. Although the actual 
saving for this stage is not directly 
quantifiable, the method's benefits become 
clearer in the versions produced for the second 
and third phases. The development times 
required are summarised in Table 1. 



The second phase of the software involved 
major task changes in the way the real system 
was used. Division of work in the processing 
centres meant trainees now no longer 
completed many of the fields they were taught 
to complete in Phase I; other fields were added 
and the format of some was altered. Different 
ways of completing input fields largely meant 
changes to navigational definitions and to 
event markers. Around twenty minutes of 
changed material in the course required thirty 
man-hours. 

Phase III involved the addition of two 
completely new menu options. Although the 
use of these on the real system did not involve 
much data Input, the menu from which they 
were accessed changed and would have 
Involved a large number of cosmetic changes in 
a non-structured course. Similarly, screen 
appearances changed appreciably as did the 
allocation of function keys and field 
structures. Changes for this release, therefore, 
involved mainly changes to operating 
definitions. As the live system was now stable, 
client trainers requested a mastery test 
consisting of an exact reproduction of the 
system with trainees expected to complete a full 
data entry transaction with 'real' messages for 
tasks completed correctly and appropriate 
remediation when it was felt that stored 
answers deviated too far from correct working 
practice. The f oolkit of operating and 
navigational definitions already assembled 
allowed this, and the system changes, to be 
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completed very rapidly. Approximately thirty 
minutes of new course were created in thirty- 
two hours and delivered within a week. 

Concluding Remarks 

The first project proved the worth of CIRCUS as 
a strategy. We have gone part of the way to 
implementing it as a PC/Pilot toolkit but this 
is a matter of convenience rather than a 
requirement: the techniques of partitioning 
courses in this way can be used with many 
languages and ran, no doubt, be used with 
development strategies other than 
instructional prototyping. The motivation for 
developing these techniques came from a need 
to produce CBT for rapidly changing computer 
systems but there have been two important 
'spin-offs'. 

It was felt initially that CIRCUS could be 
applied only to CBT for computer systems 
where the course followed the same structure as 
the real system. However, separating text, 
graphic overlays and user input and co- 
ordinating them from a control module has 
been beneficial in other types of course: firstly 
courses designed to run with a variety of 
graphic cards where the appropriate images 
and fonts are called depending on what the 
delivery system has available and, secondly, in 
multilingual courseware. A course developed 
with English, French, German, Italian and 
Spanish versions was created using the 
CIRCUS methodology. By keeping text separate 
(including all text used to caption graphics) and 
treating pattern matching sequences for 
student input as operating definitions, the 
translation and production of different target 
language versions was speeded considerably. 

A benefit exists outside the training area also. 
The rapid prototyping afforded by CIRCUS 
allows real system designs to be understood by 
larger numbers of staff than would be the case 
with system or functional specifications. 
Changes to users' requirements, or perceptions 
of requirements, are consequently identified 
before it is too late, and too expensive, to do 
something about it. 

References 

Alessi, S.M. (1988) Fidelity in the Design of 
Instructional Simulations, Journal of 
Computer-Based Instruction, 15(2), 40-47 

Boehra, B.W. (1976) Software Engineering, IEEE 
Transactions on Computers, C-25 

Bryant, R.A. (1990) Automating Code 
Generation for Computer-Based Training 



Development, Educational Technology, 30(1), 
20-24 

Chen, J.W. and Shen, C-W. (1989) Software 
Engineering: A New Component for 
Instructional Software Development, 
Educational Technology, 29(9), 9-15 

Friedler, Y. and Shabo, A. (1991) An Approach 
to Cost-Effective Courseware Developmemt, 
British Journal of Educational Technology, 
22(2), 129-138 

Janossy, J.(1986) When Solutions Already 
Exist: Lessons from Software Engineering, 
Data Training, 8, 42-44 

Jay, J., Bernstein, K. and Gunderson, S. (1987) 
Estimating CBT Development Times, US Army 
Research Institute 

MacKenzie, D. (1987) Instructional 
Prototyping: A CBT Development Strategy, 
Proceedings, Learning in Future Education 
Conference, University of Calgary, 157-163 

MacKenzie, D. (1990a) A human approach to 
computer-based course development: 
instructional prototyping. In Farmer, B., 
Eastcott, D. and Lantz, B. (eds), Aspects of 
Educational and Training Technology XXIII, 
Nichols, N.Y., N.Y., 220-225 

MacKenzie, D. (1990b) An Expert System 
Approach to Authoring Tool Selection, 
Educational Technology, 30(12), 20-26 

Merrill, M.D. (1985) Where is the Authoring in 
Authoring Systems, Journal of Computer- 
Based Instruction, 12(4), 90-96 

Sampath, S. and Quaine, A. (1990) Effective 
Interface Tools for CAI Authors, Journal of 
Computer-Based Instruction, 17(1), 31-34 

Wasserman, A.I. and Shewmake, D.T. (1985) 
The role of prototypes In the User Software 
Engineering (USE) methodology, in Hartson, 
H.R. (ed.) Advances in Human-Computer 
Interaction, 191-210, Ablex, Norwood, N.J. 



9 

ERIC 



351 

3(iS 



Selected Formal Papers From the 



Special Interest Group for 
PLATO Users' Group 
(PUG) 




9 

ERIC 



I NEED SOME HELP 



Leslie Smith 



How many times have you stood In front of a 
class and offered your best explanation of a 
lesson only to turn and look into a group of lost 
faces all bearing blank expressions? Better yet, 
how many times have you had to parade 
around the room, from student to student, 
offering individual assistance to each; in 
response to a steady chorus of "I need some 
help"? As educators, we have learned that no 
matter how thorough a job we think we have 
done, no matter how precise we have been, 
there will always be students in our class, who 
at the conclusion of the day's explanation of 
the lesson, will inevitably need some help. 
Many times it is the same student over and over 
again; however, every now and then, it is a 
different student, who for some reason or 
another, just didn't understand a particular 
aspect of the assigmnent. Finally, have you 
come up with a cure for the students who are 
absent, and when they return, they not only 
bother the teacher, but they bother their 
classmates as well? 

It was with these circumstances in mind that 
17 teachers from the Chicago and East St. Louis 
area gathered in the sweltering heat of the 
summer of 1990 at the University of Illinois at 
Urbana for two weeks of intense study for a 
pilot program called NovaNet. Each teacher 
came with one common goal: to learn a 
computer programming system that promised 
to improve the educational skills of every 
student in the school system. Many of us were 
apprehensive because we were what you would 
have classified as computer illiterates; and the 
mere sight of the monitors and keyboards 
struck fear in our hearts. As much as we hated 
to admit it, it baffled us as to how we were going 
to be proficient at using a system such as 
NovaNet into our everyday scheme of things; 
when we weren't even certain we could I jarn 
how to use it yet alone master it in two weeks. 
We were, however, willing, eager and ready not 
only to stick ourselves out there, but be 
sacrificial lambs. Yes! Sacrificial lambs. The 
success or failure of this experimental program 
depended on us and how well we could get the 
information across to our students. All our 
fears, doubts and reservations were put to rest 
as we gathered with a well-equipped group of 
professionals from CERL (Computer- based 
Eduation Research Laboratory), who knew the 



system as well as many people know their 
names. The staff of CERL explored every 
possible use of NovaNet with us, and explained 
how they felt it could be best used in our 
classroom. 

The three teachers from East St. Louis Senior 
High School decided to incorporate the 
NovaNet Curriculum into ou-- everyday 
curriculum as an aid. By aid, I mean we would 
let the students use it primarily to reemphasize 
major parts of the lesson or clarify points that 
may have been misunderstood. As the 
mathematics teacher, it was a welcomed relief 
to get the system in my school. I'm one of those 
teachers who tries to reach every student in the 
class and at some point lets him realize that he 
has mastered some part of the lesson; however, 
having 30 to 35 students per class, each on 
different levels, it became increasingly more 
difficult to do this. Many times, when the 
smarter students had comprehended, I'd look 
and see 15 to 20 puzzled looks; and if I taught 
until the slower students caught on, the 
smarter ones' minds would wander off onto 
other things. NovaNet solved this problem and 
allowed me to teach to the middle of the class 
without feeling guilty. The more I have used 
NovaNet, the more advantageous it is for me to 
use the system. 

There are several advantages as well as 
disadvantages to using the NovaNet system. 
The system is absolutely superb for 
individualized instruction. The teacher can 
examine the various lessons in a given 
curriculum and with very little difficulty give 
each student his or her own personal 
prescription as it pertains to a given lesson. If 
the teacher does not wish to use this approach, 
everyone can begin in the same place, at the 
same time and proceed through the lesson at 
his own rate. This alleviates the pressure put 
on the slower learner to keep up and relieves 
the boredom gotten by the faster students when 
they have completed the assignment and are 
waiting for the teacher to go on to the next 
assignment. Throughout the individualized 
lessons, the teacher can place off-line 
assignments which can be used for grading 
purposes. 

Off-line assignments are assignments 
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programmed into the computer by the teacher. 
They can be assigned pages out of a textbook, 
handouts from a workbook or board 
assignment, etc. Off-line assignments are not 
the only means of grading which can be 
implemented with NovaNet. Many of the 
various lessons and modules have quizzes 
contained within them. Also many of the 
lessons will give a grade based on the number of 
questions answered correctly. NovaNet Is 
equipped with a testing and graphics program, 
which allows the teacher the freedom to create 
his/her own test, called Testing 1-2-3. 

Testing 1-2-3 Is more than a test making 
program. It Is an excellent tool for graphic 
design. Many of the handouts available today 
were written or created using the graphics 
portion of Testing 1-2-3. 

Another major advantage of the system is the 
manner in which it handles right and wrong 
answers. For example, after the student keys In 
the right answer, such words of praise as: 
GREAT! SUPER! O.K.! TERRIFIC! and GOOD! 
will appear on the screen. These phrases not 
only make the students feel good about 
themselves, but they Inspire them to attempt to 
work more problems in the lesson. Wrong 
answers are followed by such expressions as: 
INCORRECT! NO! and TRY AGAIN!* For each 
wrong answer, students are given a second and 
in many cases a third opportunity to get the 
correct answer. After several wrong attempts 
the correct answer, along with a procedure or 
an explanation is automatically displayed on 
the screen. There Is a HELP key available 
which will many times give directions and 
explanations for obtaining right answers at 
any point during the problem- solving portion 
of the lesson. Finally, wrong answers are 
followed by several more problems of similiar 
nature to give the student more practice and to 
reassure the student that he/she has grasped a 
particular concept. 

Among the disadvantages is the fact that many 
times, the students fail to sign out correctly and 
a days worth of work could be momentarily 
lost. The students quickly find the sign-ons 
necessary to find the games section of the 
programs, and If the teacher is not doing some 
supervising, they will find that some students 
have wasted valuable time doing nothing more 
than playing games. Another disadvantage is 
that in some cases the students get frustrated 
because the computer may want them to be 
more specific than they are accustomed to 
being for the teacher. There are several other 
minor problems such as temporary shut downs 
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and phone wire problems, but these arc so few 
and far between that they shouldn't really 
matter. 

Not only does the system give welcomed relief 
to the statement "I need some help" which came 
from the students, if offered me help as well. 
Many of the lessons are graded; therefore, it cut 
down on much of the paperwork I'd have after 
giving homework. If the lessons aren't graded, 
they occupy the students In a positive way and 
that allows me time to catch up on my other 
paperwork. The Student Grade Report Page, 
which is part of the Instructors Resource 
Program, allows me, at my convenience, to 
check grades, progress and amount of time each 
student has put into each lesson on any given 
day. 

The NovaNet system has several key features 
that make it more complete than other 
computer systems. Because everything runs off 
everyday telephone lines, there is not a bunch 
of software (disk) to cany around, keep up with, 
and hope it doesn't get erased. The HELP button 
which is on the keyboard makes for easy 
assistance at any point In the program. 

I have witnessed smiles on faces that thought 
math was the worst thing since strained 
spinach. I have seen slower learners who didn't 
understand the assignment spend countless 
hours in front of the monitor eagerly awaiting 
the next problem. Students who had given up 
on mathematics now explore areas that I would 
never have been able to cover in a regular 
classroom setting. Students who were always 
the ones being helped are helping each other 
grasp concepts and find short-cuts to problem- 
solving that only using this sytem could have 
provided. No longer do I stare Into blank faces, 
wear out the soles of my shoes or feel as though 
I've let some student down. With NovaNet, no 
longer do I need any help! 
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WORKING ALONE VS. WORKING IN TEAMS DURING COMPUTER 
BASED INSTRUCTION IN PUBLIC SPEAKING 



Michael R. Hemphill Ph.D., University of Arkansas at Little Rock 
Christina C. Standerfer, University of Arkansas at Little Rock 



Abstract 

This study was designed to examine the efficacy 
of CBI in speech education- -both as a deliverer 
of content and a facilitator of interaction 
among students working as a team. Results 
Indicated Improved learner performance when 
participating in CBI with another student. 
Additional analysis revealed effects of learner 
attributes on the Impact of team-based CBI. 

Rationale 

The advantages of computer-based instruction 
are well documented and far-reaching. 
Students with diverse preparation and abilities 
can proceed through computerized lessons 
without the anxiety of being too far behind or 
the boredom of being too far ahead of the rest of 
the class. Since CBI Is self-paced, it also allows 
for efficient learning in that large groups of 
students may be exposed to virtually 
individualized instruction without intensive 
use of human instructors. Because of these 
advantages, CBI use is rapidly increasing In 
arious educational arenas, from Geometry 
(Hativa, 1984) to English (Thompson, 1990). 

It would seem large Introductory courses in 
speech communication could also benefit from 
CBI, particularly in teaching public speaking 
mechanics. However, little evidence exists 
indicating CBI is being used successfully in 
basic speech courses (Hemphill & SU nderfer, 
1987). 

The reason C73I is not prominent in speech 
education is two-fold. First, much of speech 
education is behavior-oriented, the object 
being to change how students perform some 
r-nimunicative task. CBi is loo often viewed as 
a drlll-and-practice too! to teach basic 
information, e.g., multiplication tables, 
vocabulary words, geographical locations, etc. 
This vision of CBI is falsely limited. In fact the 
dynamic nature of CBI makes it well-suited for 
speech education (Hemphill & Standerfer, 
1986). Second, much of speech education is 
"other" oriented. Speakers do not become better 
by delivering speeches to a mirror, but rather 
by standing in front of an audience. The 
computer (even In its best anthropomorphic 
guise) is still an Inanimate data processor. 



However, the computer can provide the speech 
teacher with educational advantages by serving 
as a model of a human communicator through 
"interactions" with the student. More 
importantly, through appropriate lesson 
design and task structure, the computer may 
serve as a facilitator of interaction among 
learners working in teams. 

Teamwork at a computer terminal Is not a new 
idea. Reports of how effective it might be in 
Increasing sti nt learning are mixed. Dossett 
and Hulvershorn (1983) reported that peer 
training via CBI can be useful in decreasing 
training time, although they report no 
significant differences in achievement among 
groups receiving peer instruction via CBI, 
individual CBI instruction, or conventional 
classroom Instruction. Webb (1985) also found 
no significant differences in achievement 
between students working alone at a computer 
and students working in pairs. However, when 
computer-aided cooperative learning has been 
compared to traditional methods, some 
differences in recall of information have been 
found (Rocklin, O'Donnell, Dansereau, 
Lambiotte, Hythecker, & Larson, 1985). One 
study found thai students working in teams 
demonstrated more and better daily work than 
students working alone and showed greater 
problem-solving skills (Johnson, Johnson, & 
Stanne, 1985). 

Obviously, teamwork at a computer increases 
interaction among students. Students are 
required to interact on a minimal level to 
decide a division of labor, to indicate when 
each is ready to move ah^ad in the lesson, to 
decide on how to answer embedded questions, 
etc. If the computerized lesson is constructed 
with explicit instructions for student 
collaboration, then the interaction becomes 
more substantively related to the lesson. 

Despite previous reports that Indicate off-task 
(or social) interaction may interfere with 
completion of the task during collaborative 
learning (Dalton, 1990), one goal of CBI 
teamwork in speech education is to promote 
interaction beyond task accomplishment. That 
is, the lesson should provide the students with 
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an opportunity to discuss the mechanics of 
communication as well as provide an 
opportunity for off-task discussions among 
team members. In this way the lesson, by 
encouraging social interaction, may promote 
positive communication experiences during 
the lesson and positive attitudes about 
communication in general. At the same time 
the lesson, by requiring task interaction, could 
be expected to increase the students* knowledge 
concerning effective communication. 

However, such an interactive-intensive team- 
based approach may not work for all students. 
Webb (1985) suggested that differences in 
prerequisite skills may moderate results when 
comparing individual and team CBI 
achievement scores. Dalton (1990) found that 
team performance may be influenced by the 
gender of the team members and suggested 
existing attitudes toward the subject matter 
could influence achievement. 

Student attitude toward the subject matter is 
especially critical in communication 
education. Not every student looks forward to 
the activities and exercises associated with an 
introductory speech course. Students vary in 
terms of how apprehensive th«5y are in 
communication situations and their 
apprehension could be expected to affect their 
learning. Simply put, students who are highly 
apprehensive about public speaking will react 
differently to a public speaking assignment (or 
a computer lesson designed to explain one) 
than students who are not apprehensive about 
speaking in public. The question ultimately 
becomes: 

To what extent does team-based public 
speaking CBI impact student learning and 
attitudes about public speaking? 

METHODOLOGY 

Subjects 

The subjects for this study were 40 students 
enrolled in an Introductory speech course. This 
course is a general education requirement for 
all degree-seeking students and requires a 
public speaking performance. Computer-based 
instruction is a regular component of this 
course. 

Because previous research has indicated that 
students with high and low levels of 
communication apprehension respond 
differently to highly structured 
communication tasks, such as the CBI used in 
this study (Booth-Butterfield, 1986), subjects 



were screened prior to participation using the 
PRCA-24 (Richmond & McCroskey, 1985). The 
PRCA-24 contains 24 statements assessing 
students' apprehensions about interpersonal 
communication situations, group 
communication situations, meetings, and 
public speaking and results In a summed 
overall measure of apprehension. Only 
students with scores representing moderate 
overall apprehension were included in the 
study (Richmond & McCroskey, 1985). 

Independent Variable 

Students were assigned to complete a CBI 
lesson defining and explaining seven criteria 
of evaluation (e.g., "Use an opening attention 
getter") to be used on their public speaking 
assignment. Twenty-two students were 
randomly assigned to work in pairs on the 
lesson. In an effort to promote interdependence 
between the two team members (Dalton, 1990), 
the team had to collaborate on seven exercises, 
one for each of the criteria. The lesson also 
included prompts before the description of each 
exercise to remind the subjects to work 
together. 

The remaining 18 students were assigned to 
work on the lesson by themselves. The subjects 
in the "individual" condition completed the 
lesson and the exercises on their own. 

Dependent Variables 

After completing the computerized lesson, 
subjects were required to list the seven criteria 
of evaluation and complete a 21 -item test over 
the 7 criteria. Student learning was evaluated 
be assessing the number of criteria correctly 
identified and the number of test questions 
correctly answered. In addition, after 
completing the lesson, subjects filled out a five- 
item measure adapted from the STAI (A-State) 
anxiety scale (Spielberger, Gorsuch & Lushene, 
1970) assessing how tense, calm, relaxed, at 
ease, and Jittery the subject feels about an 
upcoming speech. 

Results 

Those subjects participating in the team-based 
CBI were able to cite more of the seven criteria 
of evaluation (x = 5.65) than those working 
alone (x = 4.41, t = 2.07, df = 38, p<.05). However 
there were no significant difference3 in either 
test scores or attitudes about the upcoming 
speech. 

Because the PRCA-24 assesses levels of 
apprehension in four different contexts, it was 
decided to further explore the effect of team 
composition on performance and attitude by 
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examining more closely each of the four types 
of apprehension. Subjects were divided Into 
two median-split groups (low and high 
apprehension) and separate 2 (team condition) 
x 2 (type of apprehension) ANOVAs were 
conducted for each type of apprehension. Any 
Interaction between team condition and type of 
apprehension would provide evidence of an 
important learner characteristic that mediates 
the effect of teamwork on performance. 

Once again, only those analyses involving the 
subjects' ability to list the seven criteria of 
evaluation correctly yielded significant 
findings. The only significant interaction 
involved group communication apprehension. 
Those students low in interpersonal 
apprehension identified significantly fewer of 
the criteria when working by themselves (x = 
3.22) than those students working in teams (x = 
5.75; x = 5.50) and those students high in 
Interpersonal apprehension working by 
themselves (x = 5.82, F = 6.72, df = 36, p<.02). 

Discussion 

This study was designed to examine the efficacy 
of CBI in speech education- -both as a deliverer 
of content and a facilitator of interaction 
among students. It was thought that the task- 
oriented discussion among teammates would 
produce better results on testing and social- 
oriented discussion would improve the 
students' attitude about their upcoming speech. 

Partial support was found for Improved learner 
performance when participating in CBI with 
another student. Students who worked in 
teams were able to recall more of the 
evaluation criteria than those who worked 
alone, reinforcing previous research on 
collaborative learning (Johnson, et al. 1985; 
Rocklln, et al., 1985). However, no Impact on 
student attitude about the upcoming speech was 
found. 

In examining the Impact of specific learner 
attributes it was found that students low in 
interpersonal apprehension (meaning they are 
very comfortable in one-on-one situations) 
performed poorer than the other students when 
left to work by themselves. At the very least 
this supports Webb's (1985) and Daltoris (1990) 
contention that learner attributes can Interact 
with collaborative learning. It should be 
remembered that only those students whose 
overall apprehension level was moderate 
participated in this study. Thus, more extreme 
effects may be expected with students with 
greater or lesser levels of apprehension. 
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The area of student apprehension as it relates 
to CBI needs to be explored more fully, 
especially In speech education. Teaching is a 
communication process and student 
apprehension about communicating with 
others will impact that process. CBI, to the 
extent it models or facilitates interactions with 
students may generate the same type of student 
anxiety. Thus, speech educators need to be more 
aware of student apprehensions before deciding 
to pair them with another learner. 

On the other hand, CBI may prove to be an 
effective means of reducing anxiety within 
students. While research abounds in speech 
education concerning traditional techniques to 
alleviate speech anxiety, no research has been 
conducted to examine CBI's effectiveness in 
alleviating anxiety. Do students view the 
computer as a non-threatening, non- 
evaluating source of information? Can lessons 
be structured In such a way as to deliver content 
aimed at reducing anxiety while at the same 
time giving the student a chance to practice 
anxiety-reducing techniques with a partner? 
Will off-task discussions among highly 
anxious students (as opposed to the moderately 
anxious students participating in this study) 
help reduce anxiety and improve student 
attitudes about future communication 
experiences? 

This study uncovers some of the potential of 
CBI In speech communication education, but It 
also suggests some exciting issues related to 
using CBI as a facilitator of interaction among 
student learners. Once again the conclusion 
seems to be that the use of computers in 
education is not in and of itself a solution to a 
problem. But rather, the effectiveness of CBI is 
dependent upon an interaction of lesson design 
and learner attributes. 
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GENERATIVE PROCESSING OP STRUCTURAL KNOWLEDGE IN HYPERTEXT 



David H. Jonassen, University of Colorado 



Learning and Semantic Networks 

The interrelated knowledge within semantic 
memory enables humans to combine Ideas, 
infer, extrapolate or otherwise reason with the 
information. Learning consists of building 
new structures by assimilating environmental 
information and constructing new nodes that 
describe and interrelate them with existing 
nodes and with each other (Norman, 1976). It 
requires forming links between existing 
knowledge and new knowledge in order to 
comprehend information from the 
environment. Learning therefore may be 
conceived of as a reorganization of the 
learner's knowledge structure that results from 
the learner's Interactions with the 
environment. This hypothesis has been 
empirically demonstrated. As a result of 
teaching, the learner's knowledge structure 
closely resembles the instructor's (Shavelson, 
1974; Thro, 1978). So, learning may be 
conceived of as the mapping of subject matter 
knowledge (usually that possessed by the 
teacher or expert) onto the learner's knowledge 
structure. 

Hypertext engines or structures can be designed 
to reflect the semantic structure of a subject 
matter expert (Jonassen, 1991). The research 
question that is implied by these assumptions 
is, "if the node-link structure of the hypertext 
reflects the semantic structure of the expert, 
will the expert's knowledge structure be more 
effectively mapped onto the novice brov ^er?" 
This is the question that has been investigated 
in the studies described below. 

Semantic Networks for Designing Hypertext 

If we accept the suggestion that hypertext 
should mirror the semantic structure of an 
experienced performer or expert, then we need 
methods for mapping that semantic networks 
onto the hypertext. This may be accomplished 
by having an expert or group of experienced 
individuals or experts generate semantic maps 
using any of the techniques described above. 
The maps provide a node-link structure which 
may be mapped directly onto the hypertext. 
The concepts in the semantic maps comprise 
hypertext nodes and the relationships are in 
effect hypertext links. Being able to work 
Interactively and iteratively with an expert or 
group of knowledgeable Individuals to refine, 



clarify, and correct these structures should 
provide more meaningful and useful maps. The 
assumption of this method, that using 
semantic maps to define the structural model 
of a hypertext by directly mapping the expert's 
organization of ideas onto the hypertext, has 
been the subject of considerable debate 
(Jonassen & Mandl, 1990) and is the subject of a 
series of studies described below. 

Developing Graphical Browser from Semantic 
Maps 

The most direct way to map the expert's 
semantic structure onto a hypertext is to use 
the semantic map as a graphical browser In the 
hypertext. Graphical browsers are maps or 
graphical listings of available nodes in a 
hypertext. They represent a graphical interface 
between the user and a hypertext that is 
designed to reduce navigation problems within 
the hypertext (Jonassen, 1988). Getting lost in 
a large web of hypertext nodes and links is a 
common problem among hypertext users, so 
graphical browsers are developed to provide a 
spatial map of the organization of nodes in a 
hypertext. Most often, however, the 
arrangement or structure of nodes that are 
illustrated in a graphical browser Is arbitrary 
(eg. rows and columns of nodes). The 
hypothesis of our recent research (Jonassen & 
Wang, 1990; 1991) is that by arranging the 
nodes In a graphical browser according to an 
expert's semantic map, you are explicitly 
conveying the organization of Ideas in the 
expert's knowledge structure. That is, you are 
showing the user how the expert thinks. So, 
while navigating through a hypertext, the user 
is in effect navigating through the expert's 
knowledge structure. The research questions 
that we have pursued focus on the extent to 
which the semantic structure illustrated in 
graphical browsers actually maps onto the 
user's knowledge structure. To what extent will 
the user mode' oi replicate that structure in 
their own knowledge representations? 

Methods 

In order to assess structural knowledge 
acquisition following various treatmerts, we 
developed three subscales often questions each 
to measure different aspects of structural 
knowledge: a) relationship proximity 
Judgements, b) semantic relationships, and c) 
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analogies. All of the structural knowledge test 
questions were developed to focus on 
relationships between Important concepts 
contained in the hypertext. The relationship 
proximity judgements required that students 
assign a number between 1 and 9 to each of 
several pairs of concepts to indicate how strong 
a relationship they thought existed between the 
concepts in each pair (Diekhoff, 1983). For 
example: 

information retrieval systems and online 
documentation 

hypertext processing strategies and database 

The semantic relationships subscale consisted 
of multiple choice questions that required 
students to identify the nature of the 
relationship between two concepts. These 
relationships were paraphrased from the 
hypertext knowledge base. For example: 

unstructured hypertext navigating 

hypertext 

a. produces problems in 

b. defines the functions of 

c. counteracts the effects of 

d. enabled by 

Finally, the analogies subscale required 
students to complete 10 analogies consisting of 
four of the concepts from the hypertext. For 
example: 

accessing Information : index :: Integrating 
Information : 

a. links 

b. hypermaps 

c. idea generator 

d. multi-user access 

These questions were used to assess structural 
knowledge acquisition. In order to provide 
standards for assessment, three authors and 
researchers In the hypertext field agreed on the 
answers to each of these questions. In addition 
to assessing structural knowledge, ten lower- 
order information-recall questions were 
developed. 

Treatments 

The hypertext that was used for all of the 
studies was the HyperCard version (Jonassen, 
Roebuck & Wang, 1990) of the book 
Hypertext/ Hypermedia (Jonassen, 1989a). 
This hypertext is a browsing system consisting 
of 240 cards and 1167 links in three stacks 
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supported by bookmarklng and limited 
annotation capabilities. All treatments In all 
studies contained embedded referential links 
in the cards. Terms in the text were 
highlighted, enabling learners to immediately 
traverse the links. The treatments varied in 
terms of the types of browsers that were made 
available and the ways that they depicted 
structural information. Each of the 75 major 
concept nodes contained a main "related terms" 
card, which was the first card accessed when 
traversing a link to that node. 

One of the weak*. ,sses of the previous study 
(Jonassen & Wang, 1990) was the lack of 
instructional support provided the learner. 
Hypertext is a technology that supports 
Information retrieval and search tasks. 
However, these tasks are not necessarily 
correlated with instruction. One of the most 
significant potential problems in learning 
from hypertext is Integration of the presented 
information into knowledge structures 
(Jonassen, 1989a). Learning requires that 
users not only access information but also 
Interpret it by relating it to prior knowledge. 

Therefore, in this study, we attempted to 
provide additional instructional support by 
including a more generative form of treatment. 
Generative learning occurs when learners 
relate information meaningfully to prior 
knowledge (Wittrock, 1974). The control 
treatment was compared with the pop-up 
treatment and the generative processing 
treatment. In the control group, each related 
terms card provided a list of terms that are 
related to the concept being currently 
exam, d. This list provided links but no 
struct mal information about the nodes or 
links. Hie experimental treatments replaced 
these lists with a pop-up window which 
described the nature of the link (one of 12 
different link types) being traversed. The 
generative treatment was similar in 
appearance to the pop-up window version of the 
same hypertext used In the previous study. 
However, rather than being told what the 
nature of the relationship was between the 
nodes being linked, the learners were required 
to classify the nature of each link themselves. 
The pop-up window presented 12 different link 
types and required the learner to determine 
which of the link types most accurately 
depicted the nature of the relationship Implied 
by the link that they were traversing. The user 
had the option of returning to the preyious 
node or moving forward to see the node they 
had selected as many times as necessary. 
Knowledge of results was provided for each 
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selection by the user until the user selected the 
correct link type. 

The studies Involved undergraduate pre-service 
teachers in a teacher education program, who 
were assigned to learn about an important new 
instructional technology, hypertext, as an 
assignment in a pre-service instructional 
technology course. Students individually 
Interacted with and studied the hypertext for 
one to two hours in order to acquire as much 
information about this new technology as 
possible. A monitoring program was added to 
the stacks to audit the learner interactions. 

Results 

As in the first study, the recall scores of the 
control group (no structural information 
provided) were higher than either oi the two 
structural treatments. No differences between 
relationship proximity judgements, semantic 
relationships, or analogy scores occured. 

In this study, we also assessed the Individual 
difference characteristics, field Independence 
(Hidden Figures Test) and global reading 
ability/vocabulary (Extended Range 
Vocabulary Test, both from the Kit of Factor 
Referenced Cognitive Tests (Ekstrom, French, 
& Harman, 1976)). In the control and 
generative treatments, field independent 
students performed better on the relationship 
judgements, but they were impeded by the pop- 
up version. On the semantic relationship task, 
the opposite occurred. Field dependent 
students in the pop-up version outperformed 
field Independents while they were impeded in 
the control and generative groups. No 
significant interactions occurred on the 
analogy task. Nor were there any interactions 
of treatments with vocabulary scores. 

Discussion 

To the extent to which learners attended to the 
structural cues provided in the hypertexts, they 
did acquire some structural knowledge. The 
post hoc regression analyses showed that 
students were in fact attending to some of the 
structural information. The number of 
structural cards accessed appeared to predict 
structural knowledge acquisition in the form 
of relationship Judgements in two different 
forms. The time spent with the structural cards 
suggested a relationship between accessing 
those cards and the level of structural 
knowledge acquisition. These are weak effects 
at best, though with longer exposure to the 
hypertext and the prior acquisition of 
hypertext processing strategies, these findings 
may be substantiated. 



The most questionable assumption of our 
hypotheses, based upon the results, is that 
merely providing structural cues in the user 
interface will result in structural knowledge 
acquisition. The superiority of control 
treatment subjects, who produced the highest 
level of recall, is consistent with research that 
shows that without cueing or practice, learners 
tend to recall micropropositions more readily 
than macropropositions. The structural 
information clearly impeded the recall of 
specific facts by the students in the structural 
knowledge treatments. However, it did not 
generally result in greater structural 
knowledge acquisition. Merely showing 
learners structural relationships is probably 
not sufficient to result in encoding. Requiring 
structural knowledge outcomes, rather than 
orientations, is able to produce enhanced 
acquisition of structural knowledge though. 

According to Whalley (1990), the most natural 
mode of studying hypertext is browsing. The 
question is the extent to which unconstrained 
browsing can support instructional goals. The 
data and the comments from students in these 
studies showed that they lacked a clear purpose 
for studying the hypertext. That lack of clear 
purpose manifested itself in consistent 
performance across groups, especially with the 
structural knowledge elements. It is becoming 
Increasingly obvious that learning from 
hypertext must rely on externally imposed or 
mediated learning tasks -- that merely 
browsing through a knowledge base does not 
engender deep enough processing to result In 
meaningful learning. 

Hypertexts are obvious information retrieval 
technologies. However, retrieval of 
information is not sufficient by itself to result 
in meaningful learning. When the goals of 
accessing information require deeper 
processing, then deeper processing is more 
likely to occur. However, simply browsing 
hypertext is not engaging enough to result in 
more meaningful learning. It may well be that 
hypertext is not be very appropriate for highly 
structured learning tasks, as Duchastel (1990) 
suggests. 

Students in this study appeared, as expected, to 
lack hypertext processing strategies (Jonassen, 
1989), which likely precluded the most effective 
use of the technology. Hypertext, and the 
greater learner control of instruction that it 
entails, is a novel form of instruction for these 
learners. The more novel the appearance of the 
hypertext (te. graphical browser version), the 
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more negatively the students reacted to it. A 
fair evaluation of learning from hypertext can 
only come from hypertext-literate learners 
who have developed a useful set of strategies for 
navigating and integrating information from 
hypertext. 

As expected, individual differences interacted 
with learning from hypertext. Field 
independent processors generally prefer to 
impose their own structure on information 
rather than accommodate the structure that is 
implicit in the materials. In the second study, 
field independent learners were Impeded by the 
structural information provided In completing 
the recall task. However, on structural 
knowledge outcomes, they were the only 
learners able to successfully use the structural 
cues to acquire more structural knowledge 
information than field dependent learners. It 
Is likely that field independent learners are 
better hypertext processors, especially as the 
form of the hypertext becomes more referential 
and less overtly structured. 
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EXTERNAL EVALUATION OP POSIT 



Michael A. Orey, The University of Georgia 



Abstract 

While there has been a great deal of expectation 
and a small amount of evidence for the benefits 
(Anderson, Boyle, & Reiser, 1985; Raghavan & 
Katz, 1989; Kurland, Granville, & MacLaughlln, 
1990) of Intelligent Tutoring Systems (ITS), 
there Is at least as much equivocating evidence 
(Center for the Study of Evaluation, 1986; 
Sleeman, Kelly, Martlnak. Ward, and Moore, 
1988; vSleeman. et.al., 1989; Stasz, 1988). It Is 
my Intent to present some of the findings from 
an external evaluation of POSIT which would 
add to the support. Llttman and Soloway (1988) 
suggest that evaluation of ITS has two facets: 
external and internal evaluation. External 
evaluation addresses the question, "What Is the 
educational Impact of an ITS on students?" (p. 
209). Orey, Legree, GUlls, and Bloom (1991) 
suggest that external evaluation may be most 
appropriately conducted for ITS by comparing 
the effectiveness of the ITS to that of human 
tutors and regular classroom Instruction, 
although there are some dangers In using these 
groups for comparisons (Clark, 1983; Hogan, 
1958; Reeves, 1986). Some effort has been 
extended toward defining these groups in terms 
of emerging constructs as observed from video 
tapes. This paper reviews some of the external 
evaluations that have been conducted on ITS 
architectures. It then details an evaluation of 
POSIT (Orey, & Burton. 1990), an ITS for the 
tutoring of whole number subtraction. This 
evaluation used the three groups: computer 
tutor, human tutor and regular classroom 
instruction. The results, although somewhat 
limited, indicate that there is some support for 
ITS systems. 

External Evaluations: What's been done 

Although over twenty ITSs have been 
developed, very few of these systems have been 
externally evaluated. Many of these systems 
were developed on AI workstations that cannot 
be used to economically distribute training. 
These systems were primarily designed to 
demonstrate the types of interactions and 
approaches that could be used to deliver 
training. The fact that these ITSs were not 
intended for widespread use may account for 
the paucity of formal evaluations. 

Much ITS that is now being developed will run 
on microcomputers because of improvements 



in PC based software. These systems, although 
expensive to develop, have the potential to 
become cost-effective training systems because 
the computer programs will run on readily 
available microcomputers. However, because 
microcomputer based ITS remains expensive to 
develop, It is critical that performance data be 
collected to determine its effectiveness. 

In the past, the developmental cost of ITS 
technology has been Justified by the claim that 
this technology may approach the effectiveness 
of one -on-one human tutorial instruction 
(Anderson & Reiser, 1985). In fact, at least one 
source (Llttman & Soloway, 1988) has suggested 
that ITS effectiveness will eventually surpass 
the effectiveness of the human tutor. This 
should have major Implications for training 
and education because students who are taught 
by human tutors perform approximately two 
standard deviations better on achievement 
tests than students taught by traditional 
methods (Bloom, 1984). The demonstration 
that ITSs lead to similar gains in training 
effectiveness should also Justify continued 
support for developing practical 
microcomputer based ITS. 

Evaluating ITS technology with well designed 
research studies is critical to avoiding a 
repetition of the history of the evaluation ol 
conventional computer aided instruction (CAI). 
Initial meta reviews of CAI studies (Kulik, 
Bangert & Williams, 1983; Bangert-Downs, 
Kulik & Kulik, 1985; Kulik. Kulik & Shwalb, 
1986) supported optimistic estimates of the 
effectiveness of CAI and reported effect size 
estimates ranging from 0.25 to 0.42. However, 
a meta-analysis (Clark. 1985), which 
eliminated poorly designed studies from Kulik 
and Kulik's reviews, estimated a much smaller 
effect size for CAI effectiveness. 0.09. This later 
estimate underscores the importance of using 
sound evaluation procedures to estimate the 
effectiveness of microcomputer based ITS. 

The impact of ITS is also ambiguous, but 
evaluations of systems that have a wide 
curricular scope seem to indicate that ITS may 
be a powerful instructional technique. The 
MACH III, Lisp ITS, and Smithtown exemplify 
well developed systems with wide curricular 
scope. All three tutors were designed for an 
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actual training requirement and cover a large 
amount of course material. The MACK III 
supports 32 hours of Army training, the Lisp 
ITS covers a one semester course at Carnegie 
Mellon University, and Smlthtown 
corresponds to approximately one-third of an 
economics course. All three of these ITSs have 
been favorably evaluated by the demonstration 
of statistically significant group differences . 
Furthermore, the group differences were 
meaningful In that each of the tutors had a 
substantial effect on course performance. 

MACH m 

The MACH III ITS evaluation report includes 
mean and variance estimates for students 
taught with the ITS, versus those taught with 
traditional classroom Instruction. The ITS 
was used for approximately 32 hours of 
instruction and the evaluation data Indicate an 
effect size of approximately one standard 
deviation (Kurland, Granville, & MacLaughlin, 
1990). 

lisp ITS 

The Lisp ITS was evaluated twice (Anderson & 
Reiser, 1985; Anderson, Boyle & Reiser, 1985). 
In the first evaluation, groups of 10 students 
learned Lisp with either a human tutor, the 
lisp ITS, or through self instruction. During 
the first evaluation, the Lisp ITS covered 6 of 
the 16 chapters covered In the course, or 38 
percent of the course. The first evaluation did 
not demonstrate group differences on course 
exams, however, time data Indicate that the 
human tutor, ITS, and traditional Instruction 
groups required different amounts of time, 
11.4, 15, and 26.5 hours, to cover the same 
material. 

In the second Lisp ITS evaluation, two groups of 
ten students learned L'*p with either the Lisp 
ITS or traditional classroom instruction. 
During this evaluation, the ITS covered 9 of the 
16 chapters, or 56 percent of the course. A class 
exam indicated that the Lisp ITS students 
"performed 43 percent better than" the 
traditional classroom instruction group. 

Unfortunately variance estimates were not 
reported for either Lisp ITS evaluation, thus it 
is not possible to estimate effect size. However, 
the time estimates and the classroom test 
scores are consistent with the 1.0 effect size 
estimated for the MACH III. The fact that 
differences were only demonstrated on the 
knowledge test for the second Lisp evaluation is 
most consistent with the assertion that ITS 
evaluations are more likely to be successful for 
better developed ITSs. 



Smlthtown 

Two Smlthtown evaluations are described by 
Raghavan and Katz (1989). The first 
Smlthtown evaluation demonstrated that 
students who worked with the tutor spent much 
less time, 5 hours, to learn material that 
required 12 hours of time for students who did 
not have access to the tutor. This time savings 
estimate is consistent with time data from the 
Lisp ITS evaluations. 

A second evaluation study demonstrated a 15 
point advantage on a course test for students 
who had access to Smlthtown during 
instruction as compared to students who did 
not. These results are consistent with the Lisp 
ITS data, as well as with the 1.0 standard 
deviation effect size estimated for the MACH III 

Description of POSIT 

POSIT is an ITS for the tutoring of whole- 
number subtraction. The design of this system 
(see, Orey & Burton, 1990, for a detailed 
description of the design) Is based somewhat on 
Anderson's (1987) ACT* theory of learning. 
Within this theoretical orientation, it is 
assumed that the learning of a cognitive skill 
builds from declarative knowledge. In terms of 
subtraction, the algorithm to be learned is 
made of a set of declarative information. 
During the development of this skill, this 
knowledge becomes after making the Error of 
Omission (see Table 1 for error types) - 
Decrement (#1 and #2 are variables 
consolidated into the specific skill (in this 
case, proficiency with a subtraction algorithm 
develops). Declarative knowledge is provided 
to the learner In a text form when errors occur 
or when the learner asks for help. For 
example, this is a typical message that are 
bound during the execution of the program) 
"You have to complete the borrow into the ones 
place by taking away one of the tens. So, the 
correct value for this area is #1 You typed #2 
Please enter the correct value." Although this 
is a very brief description of POSIT, it should 
suffice for the purposes of this paper. It Is also 
necessary to provide a short description of the 
participants and the method used for the 
collection of data for this paper. 



Errors of Commission 

5. Increment 
4. Decrement 
11. Both 



Errors of Omission 

1. Increment (add 10) 

12. Decrement (subtract 1) 

13. Both 

7. Neither 

Errors of Creation 

8. smaller-from-larger 
10. 0-N=N 

9. 0-N=O 

14. N-N=N 

15. N-N=0 with regrouping from 

16. M-S=0, with S>M 

Table 1. The four error categories and instances of errors in each 
category (numbers are ranked according to number of occurrences). 



Other Errors 

2. Fact error 

6. Typing error 

3. Leading 0 



Data Collection 

Evaluation was conducted in the fall of 1990 in 
a lower middle class suburb of Michigan. The 
entire third grade was used. This constituted 38 
possible participants. One arrived late for the 
study and was eliminated and a second 
demonstrated mastery for subtraction prior to 
the beginning of the study which means that a 
total of 36 participants were used. The average 
age was Just over 8 years. These participants 
were split into three groups; human tutor, 
computer tutor, and regular class. This yielded 
12 participants for each group. The study was 
conducted over a five week period and the 
participants spent approximately 30 minutes a 
day working on subtraction. 

The University of Georgia, Wayne State 
University, Apple Computer, Inc. and Oakland 
Elementary School to Royal Oak, Michigan, 
collaborated to implement this evaluation. 
Both universities agreed to co-direct this 
project, provide technical expertise, and 
monitor and evaluate the program. Apple 
provided ten Macintosh SE's for a lab 
environment as well as the necessary system 
and utilities software. 

The principal at Oakland Elementary agreed to 
obtain parental permission for students to 
participate in the POSIT program, meet with 
participating teachers to facilitate scheduling 
of students in the study, collect and record 
research data on a dally basis, and facilitate 
the exchange of data between the various sites 
and the University of Georgia. An integral 
factor was the utilization of parent volunteers 
to assist in implementation of this exploratory 
study by monitoring the various student groups 
involved. Most importantly, provisions were 
made to assure that all students in any of the 
three groups (tutor group (human), computer 



group (POSIT), and regular classroom group 
(large group instruction)) were able to receive 
some computer instruction (in other curricular 
areas for the non-POSrr groups). 

Groups 

There were three groups in the study: ITS. 
tutor, and class groups. The ITS group used the 
10 Macintosh computers each with a 20 
megabyte hard drive and at least 2 megabytes of 
RAM. POSIT was available on each machine 
and it constituted the instructional treatment 
for the US group. The tutor group consisted of 
students who were tutored by parental 
volunteers in the ratio of 3 tutees to 1 tutor. 
The last group was the regular class 
instruction. Because the students came from 
two different classes and because the current 
content of the math portion of these classes 
was not subtraction, a teacher's aid was used to 
perform the instructional treatment for this 
group. The twelve students who were selected to 
participate in this group would then meet 
together as a group with the teacher's aid. All 
three groups followed a mastery learning 
approach that required the students to 
complete a mastery test (on paper generated by 
POSIT for the tutor and the class groups and via 
the computer for the ITS group) with 2 or fewer 
errors out of 18 problems. 

Materials 

The ITS group used POSIT (Orey, & Burton, 
1990). The regular class and the human tutor 
groups used materials that were generated by 
POSIT. POSIT uses as its curriculum 18 
problem levels (ranging from 2-digit problems 
without regrouping to 4-digit problems with 
zeros in the tens and the hundreds place of the 
minuend). Ten tests were generated by POSIT 
which were made up of one problem from each 
of the 18 problem levels. These tests were used 
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to determine mastery. For worksheets, POSIT 
generated two worksheets for each of the 18 
problem levels. These worksheets included 18 
problems at the specified level. 

Procedure 

Prior to the initiation of the study, all students 
were administered an achievement test that 
was generated by POSIT (in paper format) using 
its 18 problem levels. These tests were graded 
and the students were rank ordered according 
to these scores. Using a cluster of three, the 
students were randomly placed into one of the 
three groups. That is, the top three students on 
the list were randomly assigned to each of the 
three experimental groups. In this way, each 
group h;*d one of the top three students, and the 
next three, and the next three, and so on. 

The pretest was then used by the tutors or 
teacher to establish instructional progression. 
POSIT uses its own pretest, so on the first 
couple of days of the experiment the ITS group 
was taking another pretest. All three groups 
used the pretest to determine which probltm 
level (1-18) where each student needed practice. 
POSIT would continue to present problems at 
that level until the student was able to perform 
correctly on three problems in a row. At that 
point, POSIT would go on to the next level 
where the student had difficulty. This process 
continued until all problem levels where the 
student had difficulty were resoVed. At that 
point, the student was given a tu. for mastery. 
If mastery was achieved, the student was locked 
out of the program. If the student did not 
achieve mastery, this new test was used to 
determine which problem levels the student 
needed to work on. 

Similarly, the tutor and class group used the 
pretest to determine which worksheet the 
student needed to work on. The student worked 
on the problem level that was determined from 
the pretest until the teacher or tutor 
determined that the student might continue to 
the next level. It should be noted here that as 
the students worked on these problems they 
received feedback from the tutors or teacher. 
The difference between these groups was that 
the teacher had twelve students while the tutors 
only had three. The class size was somewnat 
smaller than a typical class size, but the size 
was determined from the fact that the 
population from which to draw was limited to 
36 students. The students in the tutor and class 
conditions continued through the problem 
levels until they had completed all problem 
levels where they had made errors on the 
pretest. At that point, they were administered a 



mastery test. If mastered, they were no longer 
removed from class for the project (to do 
subtraction). If not mastered, this latest test 
was used to determine which levels to work on. 

Results and Discussion 

The results can be described in a variety of 
forms. One important aspect of the results is 
the data that was collected during the 
evaluation which can be used to more fully 
describe each of the conditions. This data 
includes video tapes of each experimental 
condition, a record of the number of problems 
that students completed in each condition, and 
averages on the pretest. The second aspect of 
the results is that of effectiveness evaluation. 
There were a variety of measures for this 
aspect. The primary one is time until mastery. 
However, there was other data which was used 
to demonstrate effects. That other data 
includes weekly quizzes that were 
administered via paper and pencil and a 
delayed posttest that was on paper and pencil. I 
will begin with the descriptive data. 

Descriptive Data 

Table 1 shows the numbers for the total 
number of problems completed by each group 
(an ANOVA (p=0.002, F=7.85) was run on these 
numbers and the results indicate that the ITS 
group performed significantly fewer problems 
than the other two groups using Fisher Post 
Hoc methods) and the mean and standard 
deviation on the pretest for each group. The 
average number of problems completed by each 
group may be slightly misleading, after all, 
once a student achieved mastery, they did not 
do any more problems. However, it also might 
be due to the fact that POSiT did not waste time 
on determining if the child needed to progress. 
In the other conditions, this may not have been 
the case. 

The pretest was measured in the number of 
Ienois out of 18 problems. As can be seen, the 
average child did not know how to perform 
subtraction very well (on the average, each 
child got 5 out of 18 problems correct). There 
were no significant differences between the 
groups. If anything, the average child in the 
ITS condition performed more poorly than any 
other group. 

Video tapes were created for each of the three 
conditions. The categories that have emerged 
from this data are Teacher Presence (teacher in 
the general vicinity of child, not present, or 
actually tutoring the child, see Table 2), 
Teachur Interactions (motivational comments, 
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Group Average # problems Mean Pretest SD Pretest 

ITS 159 13.42 3.1 

Class 217 13.25 32 

Tutor 265 12.92 4.0 

Table 1. Descriptive data for evaluation 





Present 


Total 


Tutoring 


Tutors 








w 


18:46 


17:12 


00:20 


X 


19:48 


25:05 


01:47 


Y 


18:03 


23:10 


04:42 


Z 


16:00 


18:53 


10:36 


Tutor/Avg 


18:09 


19:45 


05:41 


Class 


05:34 


13:45 


02:45 


POSIT 


11:42 


23:44 


05:04 



Table 2. Time spent on teaching. 

immediate feedback, delayed feedback, 
support, dl-vipllne, see Table 3), and Student 
Interactions (performance comparison, help, 
Interference, see Table 4). 

As can be seen in Table 2, the tutor or teacher is 
near the child for a greater period of time under 
the tutoring condition (almost all the time - 
18:09 out of 19:45) rather than in the class 
situation (about two fifths of the time) or the 
ITS condition (about half che time). In 
addition, as you might expect proportionately 
more time to the tutoring condition is spent on 
tutoring than the other conditions. 

Table 3 summarizes the interactions that 
occurred between the instructors (teacher, 
tutors, or lab assistants in the ITS condition). 
The most frequent interaction that occurred 
was in the whole class condition where the 
teacher was correcting Inappropriate behavior. 
This type of interaction did not occur at all In 
the ITS group and only minimally in the 
tutoring condition (and usually this occurred 
when the tutor was require to leave the table). 
Another Interesting number is the fact that the 
lab assistants in the ITS condition tended to 
offer support more frequently than in the 
other conditions. The amount of feedback, both 
immediate and delayed, Is not surprising in the 
tutoring condition. 



Table 4 summarizes the interactions between 
students In each of the conditions. The ITS 
condition was the least social of the three 
(there was only a rare Interaction and it was to 
relate how well they were progressing). 
Another interesting attribute of these numbers 
is that the other two conditions tended to have 
interactions that would interfere with a child's 
work (one child would interrupt another child). 
This may have been due to a novelty effect. 
POSIT was not designed to motivate the child 
and subtraction of whole numbers is not the 
most interesting of topics. 

The intent of reporting this data was to more 
completely describe the three conditions in the 
evaluation. In the class condition, the 
interactions between teacher and child tended 
toward the correction of behavior. Also, the 
interactions between students tended toward 
disruption of work. The tutors tended to give 
more feedback and tended to have more 
interactions altogether than the other groups. 
The tutored children tended to interfere with 
each other, but there was some work related 
performance comparisons. The ITS group 
tended to relegate support to the adults to the 
room and there were no interruptions to 
performance. These are 
generalizations based on the data. The data 
was collected via video tape and may not reflect 
the actual processes that occurred during the 
rest of the evaluation, but all three conditions 
were video taped which might discount the 
differential effect of video taping. I now turn to 
the effectiveness data. 

Effectiveness Data 

There were three measures used to determine 
the effectiveness of POSIT. The first of those 
measures was the time until mastery measure. 
Table 2 shows the results for this portion of the 
evaluation. It should be noted that the 
evaluation was designed to run until all 
children to each condition achieved mastery. 
However, 2 students to the tutor condition 
failed to achieve mastery and 4 students in the 
class group failed to achieve mastery. The 
reason for this is that the study was run using 
parent volunteers. They volunteered for a 



ERIC 



. 369 

3SG 



specific time period (5 weeks). When that time 
was completed, they discontinued the 
evaluation, even though it was not completed. 
Therefore, It was decided to examine only the 
first 8 students In each group that achieved 
mastery (because only 8 had achieved mastery 
In the class condition). This has at least two 
effects on the data. One Is that It reduces the 
sample from 36 to 24, a large reduction. 
Second, the lower ability students may have 
been most effected by the treatments and they 
were eliminated from the analysis (who knows 
how long the remaining four students in the 
class condition would have taken until they 
achieved mastery). Given these limits, the 
following results were obtained. 



A 1 by 3 ANOVA was used in the analysis which 
produced close to significant main effects 
(p=0.0859, F=2.765). Although these results 
were not statistically significant at the .05 
level, I believe that they were substantively 
significant to do some post hoc analyses. A 
post hoc Fisher test indicated that the class 
significantly differed from the tutor group at 
the .05 level. Effect sizes were also calculated 
between the class and the other two conditions 
(again, this is inappropriate given the 
probability, but the small sample and the 
incomplete data indicated the analyses might 
be used to get sonv 



Motivate Immedl Delayed Support Behavior Other 
Tufew/Avg 4.5 3.75 7 3.75 3 7 



Class 



15 



POSIT 2 2 7 

Table 3. Interactions between the instructor and the student. 



Tutor/Avg 



Perform. 
6.5 



Help 
1 



Interfere 
9 



Other 
1.5 



Class 



15 



POSIT 



Table 4. Interactions between students. 

Indication of the current standing of POSIT). 
The effect size between the class group and the 
ITS group was 0.74. The effect size between the 
class and the tutor groups was 1.16. There are 
at least three points to be made about these 
results. 

The first is that the class/tutor effect size 
should be close to 2.0 (Bloom, 1984). This effect 
size may have been obtained if the evaluation 
would have continued until the end. On the 
delayed posttest, the 2 students not achieving 
mastery in the tutor condition got 13 and 3 
correct out of 18. The 4 students not achieving 
mastery in the class condition scored 12, 10, 9, 
and 3 correct (mastery, recall is 16 correct or 
better). Again, who knows how long it would 
have taken to get these children to 
achieve mastery. Another mitigating factor 
.hat might explain the lower than 2.0 effect size 
is the fact that the class size started with 12 
(much smaller than a typical class) and 



continued to diminish to the point where the 
class condition at the end of the evaluation was 
only 4 (not too different from the tutoring 
condition of 1 to 3). Finally, the regular class 
group used a mastery learning approach to 
Instruction. According to Bloom (1984), a 
mastery learning approach to Instruction has 
an effect size of 1.0. Therefore, the effect sizes 
for tutoring and ITS would be 2.16 and 1.74, 
respectively. These figures are more in line 
with those suggested by Bloom. Of course there 
can be other explanations, but these are 
important ones specific to this case. 

The second point to be made about these results 
is in regard to the 0.74 effect size between the 
class and the ITS groups. Recall that our 
estimate for effect size was about 1.0 (Orey, et 
al., 1991). The obvious explanation for this 
difference between predicted values and actual 
values is the fact that all 12 students in the ITS 
condition achieved masterv but only 8 
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achieved mastery in the class condition. Given 
this fact, I still maintain that the effect size 
estimate for ITS would still be in the general 
neighborhood of 1.0. 

The other measures actually confound these 
results further. The results of the ANOVA 
comparing posttest results showed no 
significant differences (p=0.8989, F=0.107). 
The results of the weekly quizzes can be seen in 
Figure 1. The initial low level of the ITS group 
might be explained In terms of the 
fact they were taking the online version of the 
pretest, not actually receiving instruction. The 
continuing low level of performance on the 
written quizzes might also be explained In 
terms of transfer. After all, the ITS students 
were learning and being tested on the computer 
and they would have to transfer this learning 



to paper and pencil activities. This transfer or 
lack of it, might be demonstrated via two 
anecdotal accounts. First, on a posttest that 
was completed by a student from the ITS group 
there were square boxes drawn above and below 
the numbers. This graphical representation 
actually matches the interface that POSIT uses 
to examine scratch work that children use 
when completing subtraction problems. In this 
case, theie was a positive transfer from ITS to 
written testing conditions. The second case 
was negative. One of the students In the ITS 
group received a 0 on the posttest. This student 
actually performed addition on every problem. 
If they had been addition problems, this 
student would have gotten all of them correct. 
Apparently, this student clearly understood 
that 
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when using POSIT, the operation was 
subtraction. This also throws off the results of 
the posttest analysis since everyone else in the 
ITS group did quite well on the posttest. 
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ABSTRACT 

This paper is a review of the literature on both 
knowledge acquisition and knowledge 
representation In Intelligent computer-aided 
Instruction. The review compares and 
contrasts systems that have used a variety of 
representation and acquisition techniques and 
makes specific recommendations based on the 
nature of the instructional content. 

INTRODUCTION 

According to the Quality Education Data (1989), 
76,395 of the 79,693 public schools In the 
United States have two or more 
microcomputers. Further, there are 1,596,715 
units - an average of 20.0 units per school or an 
average of 25.4 students per microcomputer. 
Computers are in the educational 
environment. The revolutionary potential of 
computers for Instructional use has been 
apparent since the computer was first 
introduced to the classroom. However, large 
numbers of computers does not guarantee 
quality of education. We need to identify why, 
how, when, and for what computers are to be 
used in schools. 

Roecks (1981) identified thirteen educational 
uses of computers, including instructional aid, 
instructional management, and computer- 
aided instruction. Jonassen (1988) divided 
computer use in the classroom into two 
categories, teaching about computers and 
teaching with computers. Teaching about 
computers includes teaching the concept of 
computer literacy, teaching of programming 
languages, and teaching of programming 
principles. In contrast, teaching with 
computers uses computers as an instructional 
tool or vehicle which may deliver and/or 
manage instruction. However, much of the 
present computer instruction Is about 
computers rather than with them (Becker, 
1986). Taylor (1980) classified computer uses 
as tutor, tool, or tutee. The computer as tutor is 
where the computer assumes the role of a device 
for delivering instruction such as tutorials and 
drills. The computer as a tool described in 
terms of utility functions such as spread sheets, 
database programs, and statistical packages. 
Instructional applications of traditionally 



tool-oriented applications are growing. In the 
tutee mode, the user teaches the computer 
rather than the computer teaches th? user. 
There is considerable evidence that learner 
control over content is not always advisable. 
Jonassen (1988) c'aimed that users do not 
usually make the best choices, but they can be 
advised to do so. 

Research findings showed that only 10 minutes 
out of a 50 minute class is used to work with 
selected Individual students (Bunderson & 
Inouye, 1987), and not more than two minutes 
is given to each student per day for Individual 
instruction (Deham & Lieberman, 1980). 
Computer-aided instruction (CAI) offers a way 
of providing individualized interaction with 
the learners. Individualizing instruction 
means the initial state of the learner is an 
important basis for prescription of content as 
well as determining instructional strategy. 
However, most of this traditional CAI 
courseware was criticized as behavioral 
oriented, low quality, ad hoc, or frame(page)- 
oriented CAI (Jonassen, 1988). 

Intelligent tutoring systems (ITS) have raised 
considerable interest among instructional 
designers and educational psychologists as to 
their effectiveness for teaching and training. 
ITS are developed from the perspective of 
multiple areas such as computer science, 
artificial intelligence (AI), cognitive 
psychology, and instructional technology. Not 
many AI programs developed to date have been 
used regularly in the classroom. Most AI 
programs are developed as research tools to 
analyze some aspect of computer-based 
learning. They are developed on mainframes 
or AI workstations making them unavailable 
to schools that only have microcomputers. 
Few programs have been completed to make 
them suitable for development costs. Most AI 
programs are not based on a pedagogy which is 
currently used to teach a subject. Furthermore, 
there has been little formal evaluation 
research done by AI researchers to measure 
how their programs affect learning and 
compare to other instructional alternatives 
(see, Orey, Legree, Gillis, & Bloom, 1991). 
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As shown In Figure 1. each major advance In 
instructional methodology has resulted from 
the Incorporation of a new learning concept 
using on a new generation of computer 
hardware. Al researchers are using AI 
programs to model learning processes and 
plausible models of learning are evolving that 
can be tested on the computer. Others are 
taking what we know about learning and trying 
to build teaching systems that interact with 
students in the right ways. Some cognitive 
scientists are studying people in natural 
learning situations. Questions that will be 
discussed here are: what kind of knowledge is 
involved?; how can the knowledge be 
acquired?; and how should the knowledge be 
represented? 



Theoretical 
Background 


Behavioral Psychology 
(Skinner) 


Constructive Cognitive 
Psychology (Piaget) 


Cognitive Science 
(Newell & Simon) 


Hardware 
Innovation 
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computP: 


4th generation 
computer 
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computer 


Instructional 
Methodology 


¥ 

Programmed Instruction 


Discovery Learning (Bruner) 
Microworlds 


ICAI 



FIGURE 1. Evolutions of instructional methodology. 
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Remember 


Verbal Information 


Declarative 


Semantic 
Network 


Use 


Intellectual Skills 


Procedural 


Rules 


Find 


Cognitive Strategy 
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FIGURE 2. Shallow versus deep processing of knowledge and instruction 
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Table 1. Manual Methods of Knowledge Elicitation 



Types of Knowledge 



KA Techniques 



Declarative 


Almost any technique 


Procedural 


Interviewing, Observation, Protocol analysis, 




Task anal, is, Scenarios or Decision analysis 


Semantic 


Concept sorting, Scenarios, Inferential flow analysis, 




Protocol analysis, Repertory grid 


Episodic 


Interviewing, Protocol analysis, Task analysis 




Scenarios, Event recall 
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INTEGRATING COGNITIVE THEORY INvD GAGNE'S INSTRUCTIONAL EVENTS 



Michael A. Orey, James R. Okey, Marshall G. Jones and Linda S. Stanley 

The University of Georgia 



Abstract 

There has been a great deal of discussion about 
integrating cognitive theory into the 
development of instruction (Merrill. Li. & 
Jones. 1990; Orey & Nelson. 1990; Wildman & 
Burton. 1981; Winn. 1989). Some of the 
approaches have been quite radical (Merrill. Li. 
& Jones. 1990; Orey & Nelson. 1990; Winn. 
1989). This paper will explore the possibility of 
integrating cognitive theory into an 
established development model -- Gagne's nine 
events of instruction. Rather than discarding 
the old theory and replacing it with the new -- a 
Popperian approach to the development of 
science (Popper. 1968) -- we suggest here that 
the field of instructional development might 
take the less radical form of scientific 
development as described by Lakatos (1978). 
This second form is more evolutionary than 
revolutionary. Hence, we will attempt to 
describe how current cognitive theory might 
evolve into our instructional development 
models. Specifically, we will describe how 
cognitive theory might have an impact on 
development decisions related to Gagne's 
events of instruction (GagnS. Briggs, & Wager, 
1988). We will use the third edition of Gagn6's 
book to describe the most up-to-date notions of 
how cognitive theory impacts on this 
development framework. Each section of the 
following text examines an event of 
instruction. Each section includes the name of 
the event, the learning process associated with 
that instructional event, and then the textual 
discussion of the current cognitive theories as 
they relate to that event. We will tend to use 
computer-based ir<-''*»on examples where 
possible. 

I. Gaining Attention. Reception of patterns of 
neural impulses (GagnS. Briggs. & Wager, p. 
182). In terms of attention, there is a great deal 
of cognitive research that can have an impact 
on this first event of instruction. To gain and 
maintain attention, one needs to examine a 
model of attention. Norman (1976) provides 
the most cohesive model of attention within 
the information processing system. An 
important component of Norman's attentional 
model seems to be pertinence. After the initial 
perception or pattern recognition, there is a 
component of memory which Norman calls 



"Memory Recognition" which accesses Long 
Term Memory (LTM). This component of 
memory functions on the basis of pertinence 
for the individual. Pertinence can be 
something as clear as the learner's name or as 
ambiguous as a scent in the air which evokes a 
sense of nostalgia. It should be noted in 
passing that this memory recognition phase is 
at a "deeper" level of processing than that 
suggested by Gagne's definition -- Reception of 
patterns of neural impulses. Gagne's definition 
describes perception, not attention. The 
question is. how does this effect the 
development of Instruction? 

Earlier attentional models focussed on the 
physical characteristics of the attentional 
stimulus (Broadbent. 1958). Such attributes as 
- the originating location of the stimulus, the 
volume of a sound, the brightness or different 
color of an image — would cause an individual 
to attend to that stimulus. The later models 
attempted to account for such phenomena as 
the ability to recognize your own name being 
spoken in a noisy room. Norman (1976) as well 
as others (ShlfTrin. 1976) suggested alternative 
models that would account for these types of 
phenomena. The result Is that attention is not 
only Influenced by loud or unique stimuli, but 
also the context of the stimulus. The 
implication for this event of instruction is that 
attention may be achieved by using relevant 
stimuli as well as strange creative sounds or 
images. Other implications of attentional 
theory relate to the maintenance of attention. 

If attention is affected by pertinence, then 
instruction that ~ placed in the cortext of 
some over-aiching framework might aid 
attention. Brown, Collins. & Duguid (1989) 
suggest that instruction should be situated in a 
context of a real task. Therefore, if you are 
attempting to teach about earthquakes, a valid 
approach would place the student in an 
observation center that simulates an 
earthquake. This approach is better at 
maintaining attention than a tutorial that 
displays screen after screen of information 
about earthquakes. 

2. Informing the Learner of the Objective. 
Activating a process of executive control. 
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(Gagne. Briggs. & Wager, p. 182). Gagne implies 
that by providing the learner with the 
objectives, the learner can invoke a cognition 
about one's own learning. Metacognition 
means that one can have conscious control 
over one's learning or thinking (Brown, 1975). 
Metacognltlve strategies include such things 
as, selecting memorial strategies for varying 
types of lists (choosing a mnemonic, for 
example), visualizing, taking notes, and 
selecting key words in a text to aid in 
comprehension. However, there is nothing 
inherent in a behavioral objective that would 
cause a learner to invoke any of these 
strategies. For example, a Gagng behavioral 
objective with its five components might be 
[Situation) Using a paper and pencti, [learned- 
capabUity:} state [object) Gagne's nine events of 
instruction taction:] by writing them [tools and 
other constraints} without the use of notes, in 
order, and without error. This statement 
establishes the goal for the student, but there is 
no indication about what if any executive 
control/metacognitive strategy should be used. 
If the intent of stating the objective is to 
activate a process of executive control then it 
might be more appropriate to inform the 
learner of the task and how they might 
accomplish that task. This might change the 
above objective to — [Situation:} Using a paper 
and pencil, [learned-capability:) state [object:) 
Gagne's nine events of instruction [action:) by 
writing them [tools and other constraints) 
without the use of notes, in order and without 
error. To do this you might use a [activate a 
process of executive control:} mnemonic device 
such as an acronym or a key word method. 
Afterall, not all learners are equally adept at 
selecting memorial strategies, so stating how 
the learner might learn may be quite 
appropriate. If the purpose of informing the 
learner of the objective is to activate a process 
of executive control, than that process ought to 
be stated explicitly ,s part of the objective. 
Certainly, learning and memory strategies 
ought to be Included as part of the fourth event 
of instruction presenting the stimulus 
material. 

On the other hand, Gagne also includes 
expectancies as a part of his learning model. It 
might be inferred that stating the objective will 
cause the learners to establish what behavior Is 
expected of them. This definition, however, is 
more related to behavioral learning theory 
than that of cognitive theory. In cognitive 
psychology (we looked in several texts on 
cognitive psychology and the root word 
expectancy was not listed in the subject index, 
Flavell. 1985; Glover. Ronning. & Bruning, 



1990; . . Dhmeister. & Nyberg. 1982). 
expectan — S might be Interpreted from either a 
prior knowledge standpoint or schema theory 
standpoint. In this interpretation, the learner 
develops expectancies about certain 
characteristics of their environment. The 
learner has come to know that when the 
instructor states the behavioral objectives, the 
instructor is Indicating a behavior that should 
be performed. This is different from a 
behavioral interpretation and several of the 
authors' of the current paper believe that the 
establishment of an expected behavior is not a 
bad thing for Instruction. The act of goal 
setting is a powerful motivator in many 
contexts (Bandura, 1986). 

3. Stimulating Recall of Prerequisite 
Learnings. Retrieval of prior learning to 
working memory- (Gagne, Briggs, & Wager, p. 
182). This event is particularly compelling 
from the cognitive perspective. In order to 
develop cognitive structures (In other words, 
learning), one must tie what is being learned to 
what one already knows. Regardless, if the 
cognitive view is constructivist (e.g.. schema 
theory) or information processing, the 
Importance of prior knowledge is vital. In the 
schema view, the stimulation of prior 
knowledge will aid in the activation of 
appropriate schemata and perhaps their 
modification as the result of learning. This 
activation of a schema aids in both the 
interpretation of events as well as the storage 
of those related memories. In the Information 
processing view, where memory Is in an 
associative network, the stimulation of 
previous knowledge identifies associations 
that need occur. In other words, tie the new 
learning to the old. Using notions such as 
situated cognition (Brown, et. al.. 1989), new 
Information can be tied to the old if the new 
Information Is provided in the context of an 
activity that the learner has some knowledge 
about. That is, if the student that is using an 
earthquake observation center simulation 
needs to understand plate tectonics at some 
point in the simulation, that information is 
provided when and where it is needed. 

This notion Is not too dissimilar from the 
notions of Schank (1991). According to the 
case-based teaching approach proposed by 
Schank, Instruction and memory is organized 
around stories. Memory is made up of stories 
and instruction should proceed by detenninlng 
what story can be told when. As the learner 
operates a simulation, information is provided 
that is relevant to the needs of the learner. 
This information Is provided as video clips or 
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cases In most of the systems that Schank and 
his colleagues at the Institute for the Learning 
Sciences have developed. 

4. Presenting the Stimulus Material 
Emphasizing features for selective attention. 
(Gagne, Briggs, & Wager, p. 182). Perhaps the 
most powerful application of cognitive theory 
to the design of instruction would be brought to 
bear on this event. Here are a few that we 
thought were important. 

Mayer's (1989) research on models would be 
effective here. According to this research, if 
you present a conceptual model to novice 
learners prior to or during the learning of the 
material, they will be better able to recall 
conceptual information and transfer this 
knowledge to novel problems. However, they 
will be less able to recall verbatim material. 

Anderson's (1987) work on the development of 
declarative and procedural knowledge would 
also be a powerful construct for this event. 
Anderson has proposed one of the few cognitive 
learning theories (as opposed to cognitive 
theories). His theory states that as people learn 
skills they progress from declarative 
verballzable information, to linking these 
component declarative pieces together to form 
procedures, and finally to a state where those 
procedures are automatized (do not require 
conscience control). 

Resnick's (1982) distinction between syntactic 
and semantic knowledge would affect the 
nature of presentation. If the intention of the 
instruction is to get the learners to perform 
some behavior, then the instruction can focus 
on the syntax of the skills. If on the other 
hand, the behavior to be performed must also 
be understood, then the Instruction should 
focus on the semantic aspects of the to-be - 
learned material. 

Collins and Loftus* (1975) network model of 
memory would effect how the material to be 
learned was organized. If the Intent is to get the 
learner to organize information as an expert 
does, then a map of concepts might be 
appropriate. Others suggest that if we organize 
information in networks, then hypermedia 
environments are natural (they are organized 
the same way that we think). This idea is 
rather controversial since most "experts" differ 
considerably in the way they organize 
information. 

Metacognition (Brown, 1975) would also play 
an important role in the design of this event. 



Afterall, this is where the learning of the 
material takes place and metaleamlng Is how 
leaners take control of their own learning. In a 
computer-based system, one could provide 
learning strategies for each piece of material to 
be learned. The designer of the instruction has 
more time and capability to examine how best 
to learn the content than the learner does. In 
addition, strategies for learning can be used as 
the content of instruction. The nature of the 
content, the organization of the content, and 
how the content interacts all may Impact on 
how the information might get transferred 
from short term memory (STM) to LTM. Other 
cognitive notions that ought to Impact on this 
event of instruction are such notions as 
encoding processes, knowledge as chunks of 
information (Miller, 1956), and parallel 
distributed processing (McClelland, 1988). 

5. Providing Learner Guidance. Semantic 
encoding; cues for retrieval (Gagnfi, Briggs, & 
Wager, p. 182). By semantic encoding, I assume 
that Gagne means elaboration, since he 
describes this process as one which allows the 
learner to move the material to LTM. It is 
generally accepted that LTM failures are due to 
the learner's inability to retrieve the necessary 
information rather than that information 
being removed from LTM. Therefore, the more 
cues that elicit recall, the more likely the 
learner will be able to recall the information or 
the better learned the material is. Spiro and 
Jehng (1990) suggest that hypermedia 
environments can help learners to make 
multiple connections to a piece of information 
("criss-crossing conceptual landscapes"). There 
are a variety of cognitive constructs that can 
guide the developer when making design 
decisions at this phase of development. They 
all revolve around the notion that the more 
cues or links that one has for retrieving 
information, the more likely it is that one can 
do it. The first construct is that of massed 
versus distributed practice. A learner who is 
presented five opportunities to work on a 
specific task on five separate days (perhaps for 
a total of 50 minutes) will be able to perform 
better than a learner who is presented with 
those same five opportunities consecutively on 
the same day (also for 50 minutes). The 
psychological explanation for this phenomena 
is that the learner who receives the distributed 
practice may bring fresh ways of encoding the 
information that the massed practice learner 
will not do. 

A second notion is that of context specific 
learning. If the material is presented in a 
variety of contexts, there will be more links to 
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the information. Encoding specificity refers to 
the phenomena that occurs when the learner is 
provided with only a narrow context. The 
learner Is unable to draw up that knowledge in 
other contexts. For example, if you learn 
material in one classroom, you will be able to 
recall the information better in that classroom 
than a different classroom. Ginsberg (1977) 
describes a child who was able to perform 
multi-digit subtraction at the end of the school 
year without error. However, at the beginning 
of the next school year the child performed all 
problems incorrectly. Upon closer 
examination, it was determined that the child 
learned to start the problem in the column 
closest to the piano, and the next year the piano 
was on the opposite side of the room. 

Other factors that might be considered when 
developing instruction from a cognitive 
approach would be interference theory. Both 
proactive and retroactive interference can 
contribute to the failure of learning. In one 
case the new material interferes with the old 
and in the other the old interferes with the new. 
This effect is widely demonstrated with 
materials that are new to the learner or 
materials that require rote memorization. A 
second consideration which is directly related 
to interference is that of frequency. The more 
times you practice the better you will be given 
that you are provided with appropriate 
feedback (Event 7). 

6. Eliciting Performance. Activating response 
organization (Gagne, Briggs, & Wager, p. 182). 
It is not altogether clear what is meant by 
response organization. The nine internal 
processes that are described on page 1 1 ( Gagne, 
Briggs, & Wager, 1988) do not include this term. 
However, he does depict a process of response 
generation that makes use of information from 
both LTM and STM. The example given is that 
of getting the learner to actually write the 
plural form of the word appendix. This would 
require the learner to retrieve from LTM the 
rule for performing this task and then applying 
the rule to this particular instance. Once the 
rule is applied, a response would then be 
generated and either voice or hand output 
would be initiated. This appears to be a 
cognitive process. 

More recent work in cognitive psychology 
tends to focus on piraess rather than product. 
The emphasis changes from what the 
observable behavior <s to what that behavior 
indicates about the process the leaner went 
through to generate that particular behavior. 
This leads to the next event. 
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7. Providing Feedback about Performance 
Correctness. Establishing reinforcement 
(Gagne, Briggs, & Wager, p. 182). Although 
reinforcement is a construct in learning 
theory, it is not a construct in cognitive 
learning theory. This does not mean that 
reinforcement suddenly does not work. It does. 
However, this paper is addressing cognitive 
theory applied to instruction and 
reinforcement is not a part, of cognitive theory. 
Feedback, on the other hand, is an issue within 
the field and there is extensive research on the 
what, when and how of providing feedback. 
Most notably the work being done with 
intelligent tutoring systems (Wenger, 1987) and 
the research area of cognitive science 
(Anderson, 1987) has contributed to this area. 
Much of this research focuses on how the 
learner is responding (their solution process) 
and providing the specific feedback to modify 
the solution process so that a correct response 
can be generated. This does not relate to 
reinforcement -- a theory-ladened term from 
operant conditioning research. 

8. Assessing the Performance. Activating 
retrieval; making reinforcement possible. 
(Gagne, Briggs, & Wager, p. 182). It is not at all 
clear what is meant by activating retrieval at 
this point of the instruction. In terms of 
learning, you are trying to determine if the 
learner has acquired the information, 
organized the information appropriately, is 
able to retrieve the Information, and/or is able 
to use the Information in a problem solving 
setting. If you take Anderson's (1987) view of 
the acquisition of procedural knowledge, 
assessment must take place as close to the 
learning (in time) as possible. In this way, 
incorrect subprocedures can be corrected before 
they become too well established (remembered). 
Some other considerations, besides those 
implicit in the above discussion, is the the 
notion of recall versus recognition. Among 
other things, research has shown that learners 
who expect to be tested in a recall format (e.g., 
essay, problem solving test) perform better 
than those learners who expect a recognition 
format (multiple choice, matching, fill in the 
blank) regardless of what format the actual test 
takes. A second consideration might be 
between reconstructive and reproductive 
processes. With reconstructive processes, the 
gist of the information might be important 
while with reproductive processes the verbatim 
recall might be Important. 

9. Enhancing Retention and Transfer. 
Providing cues and strategies for retrieval. 
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(Gagne. Briggs. & Wager, p. 182). Gagne, Brlggs, 
and Wager (1988) suggest that retention can be 
enhanced by providing reviews of the content 
at spaced intervals (distributed practice). In 
order to enhance transfer, they suggest that 
practice be provided In widely varying contexts 
(again, multiple encoding links lead to more 
retrieval links). Transfer Is widely accepted as 
a desirable outcome of learning. However, 
there are levels of transfer. For example, a 
math teacher might build the rationale that 
math problem solving transfers to other 
problem solving. Therefore, If a student Is 
accomplished at doing problem solving In the 
Algebra context (Two trains are heading 
towards each other at 60 mph and 40 mph 
respectively....) then this skill will transfer to 
other problem solving contexts (resolving 
marital problems). The research on this far- 
reaching type of transfer is quite small. 
However, it is quite possible to help learners to 
transfer to similar types of problems. For 
example, you can do a variety of motion 
problems where the trains or boats or planes 
are heading towards or away from each other 
and focus the Instruction on those aspects of 
the problem that are similar {providing cues). 

Another aspect of transfer is the use of 
conceptual models (Mayer, 1989). Mayer has 
demonstrated several times that by using a 
conceptual model for the presentation of the 
content to naive learners, the learners are 
better able to remember {retention) and 
transfer to other areas. However, we have 
already suggested that this is an aspect for 
consideration in another event -- Presenting 
the stimulus material. 

Not all of the events of instruction match up 
perfectly to cognitive theory (e.g., Brown, 
Collins, and Duguid's, 1989) cognitive 
apprenticeship or microworlds from 
constructivistic learning theory (Harel, & 
Papert, 1990), but then again not all swans are 
white. Instructional development is In a state 
of change, as it always has been, and as it 
probably (and hopefully) always will be. The 
trick Is to learn from progress, and to make the 
change process evolutionary, and not 
revolutionary. We are advocating the further 
study of new learning theories and how and if 
they are complimentary to our existing 
theories of Instructional development. By 
doing this we can learn how to make 
instruction more effective and efficient, and 
keep the field of instructional development in a 
state of growth. 
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Introduction 

The potential and effects of computer-based 
instruction (CBI) have been reviewed in many 
contexts, including elementary and secondary 
education (cf., Bangert-Drowns, Kulik, & Kullk, 
1985; Colbert, 1991; Neimic & Walberg, 1985), 
special education (cf., Male, 1988; Robinson, 
1991; Russell, 1987), higher education (cf., 
Deaton. 1991; Kulik & Kulik, 1986; Reeves, 
1991), business training (Gery, 1987; 
Hawkridge, Newton, & Hall, 1988; Johnson & 
Foa. 1990) and military training (cf., Fletcher 
& Rockaway. 1985; Montague, 1988; Orlansky 
& String, 1981). Although only the most 
uninformed or inexperienced individuals 
would claim today that CBI is a panacea for 
instructional problems in these or any other 
education and training contexts, the interest in 
and support for interactive learning systems 
continues to grow at a steady pace. 

Yet many questions about the effectiveness of 
CBI remain, some revolving around the 
interrelated issues of "learner control" and 
"motivation." Two often touted strengths of 
CBI are that "CBI allows students to be in 
control of their own learning" and "CBI 
motivates learners more than many other 
instrucJonal methods." Further, it is usually 
argued that being in control of learning and 
being more motivated both result in greater 
achievement. The purposes of this paper are to 
review the evidence for these claims and to 
make recommendations for design of CBI to 
take advantage of what we know about learner 
control and motivation. 

Definitions and Perspective 

Learner control is defined as the design 
features of CBI that enable learners to choose 
freely the path, rate, content, and nature of 
feedback in instruction. Learner control is 
contrasted with "program control," i.e., design 
features that determine the path, rate, content, 
and feedback in instruction for the learner. 
Learner control and program control are not 



polar opposites, but represent a continuum of 
learner latitude that varies widely both within 
and between different instances of CBI. 

Motivation is defined as the processes involved 
in arousing, directing, and sustaining learning 
(or some other behavior). These processes can 
be internal cognitive or emotional states (e.g., 
curiosity) or external contingencies (e.g., 
rewards). Interactive instructional methods 
such as CBI are often promoted because of their 
capacity for stimulating internal motivation 
and/or arranging the contingencies of external 
motivation. 

One possible relationship between learner 
control, motivation, and learning can be 
stated simply as follows: "giving a learner 
control of instruction motivates him or her 
to learn more" (see Figure 1). Unfortunately, 
the research support for this relationship is 
inconsistent. Steinberg (1977, 1989) after 
completing two extensive reviews of the 
literature regarding learner control summed 
up the evidence as follows: The motivation 
effects of learner control were not 
accompanied by better performance. Learner 
control sometimes resulted in greater task 
engagement and better attitudes, but not 
necessarily in greater achievement. In some 
instances learner control led to worse 
performance than computer control. While 
many students were motivated by learner 
control, others were indifferent to it. 
Aptitude and trait-treatment research 
yielded no definitive conclusions about 
learner control (Steinberg, 1989, p. 1 17). 

Anothei possible relationship between learner 
control, motivation, and learning can be stated 
as follows: "a motivated learner is able to take 
advantage of learner control options in 
instruction and thus learn more" (see Figure 2). 



LEARNER CONTROL "■^ MOTIVATION ■■► LEARNING 
Figure 1. Possible effect of learner control and motivation on learning. 
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MOTIVATION LEARNER CONTROL am ¥ LEARNING 

Figure 2. Possible effect of motivation and learner control on learning. 



LEARNER CONTROL MOTIVATION LEARNING 

Figure 3. Possible interrelationship among 
learner control, motivation, and learning. 



Research support for this relationship is not 
much clearer than that existing for the 
relationship Illustrated In Figure 1. Klein and 
Keller (1990) concluded that "The relationship 
among learning outcomes, type of 
Instructional control, performance, and 
motivation have not been adequately explored" 
(pp. 145-146). Kinzie (1990) provides perhaps 
the most ambitious theoretical analysis and 
literature review concerning this relationship. 
She concluded that "Research and theory 
suggest that the elements of learner control, 
self-regulation, and continuing motivation can 
be mutually supporting" (p. 18). She described 
and illustrated a complex interrelationship 
among the variables of learner control of 
instruction, self-regulation of learning, and 
continuing motivation and several related 
variables such as competence, personal 
control, relevance, and curiosity. Kinzie's 
analysis is more descriptive than prescriptive 
(Clark. 1989; Reigeluth, 1983). Hence, it 
provides insufficient concrete direction for the 
design of CBI to take advantage of the 
"mutually supporting" nature of tne learner 
control and motivation illustrated in Figure 3. 
A Design and Use Dilemma 
The complexity of the possible 
interrelationships among learner control, 
motivation, and learning and the lack of 
strong, consistent evidence for the 
relationships present designers and users of 
CBI with a dilemma. Proponents continue to 
promote CBI as a solution to some of our most 
persistent problems in education and training 
primarily on the basis of its presumed power to 
motivate. For example, Dowdney (1987) claims 
that schools are beginning to meet the 
challenge of reducing the dropout rate by using 
CBI to meet individual needs and motivate 
students to stay In school. Yet, as described 
above, the theoretical and research 
foundations for designing and supporting the 



learner control and motivation aspects of CBI 
are not solid. 

On the other hand, the assumptions upon 
which the claims about learner control and 
motivation are based are intuitively appealing 
ones. It seems eminently reasonable to believe 
that students who are allowed to Influence and 
control their learning environments will 
demonstrate more positive learning processes 
and outcomes than students who depend on 
others (e.g.. teachers, parents, and/or 
computers) to control their schooling. There is 
even research evidence that perceptions of 
personal control are associated with greater 
persistence, excitement about learning, and 
motivation toward the mastery of skills, at 
least for adult learners (Wlodkowski. 1985). 

It also seems reasonable (If not obvious) that 
making instruction more motivating will have 
positive effects on learning, and there is even 
theoretical support for this intuitive 
assumption (Keller, 1987a, 1987b). And yet. 
research into the effects of learner control and 
motivation in CBI has yielded inconsistent 
support for these assumptions. The remaining 
sections of this paper represent an attempt to 
glean guidelines from the existing research to 
support CBI design and implementation 
efforts. 

Learner Control Research 

In contrast to program or system control, 
learner control in CBI exists when the design of 
the program allows learners to exercise 
significant choice over aspects of the program 
such as the feedback, content, sequence, and 
pace of instruction. However, giving learners 
carte blanche to do whatever they wish in CBI 
is not universally effective. The effectiveness 
of learner control appears to be a function of 
the abilities, inclinations, perceptions, or 
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cognitive styles of students (Salomon & Garner, 
1986), as well as the level of difficulty, number 
of questions, and type of feedback Included In 
the program (Hannafln, 1984). 

That learner control is not universally 
effective should not be surprising. Not all 
learners will possess the skills and drive to 
benefit from the options presented in learner 
controlled CBI. Component Display Theory 
provides a theoretical perspective for the 
differential effects of learner control (Merrill, 
1983). Component Display Theory promotes 
learner control as a mechanism for adapting to 
individual differences, such as personality, 
characteristics, age, sex, socioeconomic 
variables, and learning styles. 

A clear understanding of the 
interrelationships among various individual 
differences and the effectiveness of learner 
control does not yet exist. For example, 
consider the personality trait, iocus of control, 
defined as a generalized expectancy of internal 
or external "attributions" for behavior and 
outcomes (Stipek & Weisz, 1981). There is 
evidence that students with an external locus of 
control will excel in high-structure situations, 
while internal students will do better in low- 
structure situations (Joe, 1971; Wesley, 
Krockover, & Hicks, 1985). However, in 
separate studies, L6i ez and Harper (1989), Klein 
and Keller (1990), and Santiago (1990) failed to 
find a strong relationship between locus of 
control as a personality trait and the 
effectiveness of learner control as a CBI design 
feature. 

Other studies investigating the relationships 
between learner control and personality 
characteristics have yielded weak results. For 
example, Kern and Matta (1988) found a small, 
but statistically significant, relationship 
between personality traits measured by the 
Myers-Briggs Type Indicator and effectiveness 
of learner controlled CAI, and concluded that 
"Designers of CAI packages should consider 
development of intelligent systems that can 
alter the style of presentation In relation to the 
student's personality" (p. 107). Rubincam and 
Oliver (1985) reported observational evidence 
for a personality variable ranging from 
"cautious" to "confident" interacting with the 
effectiveness of learner control, and concluded 
that "Computer-b ised instruction 
incorporating learner-control options may be 
made more effective if the personality traits 
that affect student choices are better 
understood" (p. 224). Unfortunately, an 
adequate level of understanding of the 



relationships between personality traits and 
learner control needed to guide CBI design has 
not been reached. 

Ross and Morrison (1989) promote learner 
control as a potentially effective and practical 
means of adapting validated instructional 
technologies to individuals. They suggest that 
"learner control is not a unitary construct, but 
rather a collection of strategies that function 
in different ways depending on what is being 
controlled by whom" (p. 29). Ross and 
Morrison propose to allow learners to select 
contextual properties of lessons such as text 
density and problem theme according to 
learning styles and preferences. They argue 
that although learners, especially low 
achievers, are not the best qualified to make 
decisions about the instructional aspects of 
CBI, they can make effective choices among 
presentational and/or contextual options. 

Hannafin's (1984) review of the literature on 
learner control concluded that learner control 
was more effective when older, more mature, 
and /or more capable learners are involved, 
higher order skills are taught, and content is 
familiar. Hannafln also advocated the use of 
adaptive strategies such as coaching and 
advisement to support learner control, 
consistency in amount and type of learner 
control within a CBI program, and provisions 
for switching unsuccessful learners to 
program-control strategies. 

Some of Hannafin's recommendations 
approach the specifications for intelligent 
tutoring systems (Poison & Richardson, 1988). 
The potential strength of intelligent tutoring 
systems (ITS) is that knowledge of a student's 
individual differences in performance, 
aptitude, personality, and other traits will 
form the basis for instructional strategies. 
However, the development of ITS has not 
progressed as rapidly as promised (Rosenberg, 
1987). Hence, this paper deals solely with 
learner control and motivation factors as 
applied in traditional CBI programs. 

Four categories of learner control strategies are 
discussed In more detail below, viz., content of 
instruction, sequence of instruction, 
instructional pacing, and feedback 
mechanisms. In addition, the research on 
learner control with advisement and adaptive 
control is briefly described. Each of these 
subsections is accompanied by a "guideline" to 
recommend how the research might be applied 
in the design and use of CBI. 
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Content of Instruction: 

According to Carrier and Williams (1988). use 
of learner control strategies for the amount of 
content presented results in wide variation in 
the amount of material seen by students, and in 
general those students who receive greater 
amounts of instructional content learn more. 
While it seems obvious that learners who 
experience more instructional content will 
learn more, there is some evidence that 
learners of differing abilities make poor 
choices concerning amounts of instructional 
content, e.g., high ability learners (who need 
less) may select more instruction and low 
ability learners (who need more) may select 
less. One goal of Instruction is efficiency, i.e., 
students should spend only as much time on 
instructional content as is needed, Ironically, 
learner control strategies sometimes leads to 
inefficiency, such as when learners select more 
elaborative information than is needed 
(Tennyson, Christensen, & Park, 1984). 

Guideline # I: 

In general it is not advisable to allow learners 
to control access to "essential content 9 * unless 
reliable, valid performance measures indicate 
that they have mastered that content Content 
defined as enriching or repetitive should be 
under learner control 

Sequence of Instruction: 

Automatic frame sequencing has been 
criticized as -brain corroding in its banality- 
(Brinkworth & Hobbs, 1982). Indeed, the most 
devastating criticism that can be hurled at a 
CBI program is that it is an "electronic page- 
turner/ Instructional design theorists 
maintain that the effectiveness, efficiency, and 
appeal of instruction will increase when 
informed/motivated students are given the 
opportunity to select and sequence 
instructional strategies (Merrill, 1983). 

Rubincam and Oliver (1985) found that 
students adopted a consistent learner-control 
strategy (test-first or instruction-first) when 
given the option. Although no significant 
difference in overall performance was found 
between the two strategies, students selecting 
the instruction-first strategy took longer to 
complete the course. Gray (1987) found that 
control of sequencing had a positive effect on 
comprehension, but not on retention, in 
sociology CBI. In contrast to most studies 
reporting attitudinal data (Steinberg. 1977. 
1989). Gray found that students given learner 
control were more negative toward CBI than 
students under program control. Gay (1986) 
found a significant interaction between 



previous knowledge and learner control 
wherein students with low pretest scores made 
poor sequencing choices and students with 
high pretest scores learned equally well from 
learner control and program control. 
Gustafson, Reeves, and Laffey (1990) found that 
learners generally selected a linear strategy to 
progress through a hypermedia training course 
averaging more than nine hours of instruction, 
even though a wide variety of instructional 
options were available. 

Guideline #2: 

Well-designed CBI will include a variety oj 
instructional options including overviews, 
statements of prerequisites and ofyectives. pre- 
tests, tutorials, simulation exercises, and post- 
tests. Although there is some evidence that 
learners are not always the best Judges of what 
kinds of and how much instruction they need a 
well-structured orientation to instructional 
options may provide them with a better basis 
for selection. Contextual and presentational 
options, if included, should be subject to 
learner preferences. 

Instructional Pacing: 

It is generally recognized that students differ in 
the amount of time required to learn a task 
(Carroll, 1963), and that giving students 
control over the pace of instruction helps them 
to master learning objectives (Bloom, 1976). It 
also seems that control by the learner of 
display time is much more desirable than 
computer control of it (Weller, 1988). Although 
timed responses of screen changes may be 
intended to keep students alert and 
concentrated on their work, some students may 
be stressed by time limitations and/or will not 
be alert enough to keep up with the pressure of 
the time limit. In either case, their frustration 
is likely to increase and their motivation may 
deteriorate (Sasscer & Moore, 1987). 

Information processing theorists believe that In 
order to be remembered, information must be 
encoded and moved from short-term memory to 
long-term memory. Short-term memory has 
limited processing capacity, so the student must 
selectively scan material in order to limit 
incoming information, and encode it into long- 
term memory after integrating the information 
(Wager & Wager. 1985). 
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Figure 4. Typical information processing model (Mayer, 1989). 



According to Mayer (1989), learning processes 
refer to the way in which students encode to-be- 
learned information. Figure 4 shows an 
information processing model for describing 
meaningful learning processes. It takes some 
time to select and organize new information, 
integrate it with existing knowledge, and encode 
into long term memory. Learners differ in the 
amount of time required for this encoding 
process (Mayer. 1989). Students should be 
allowed to take whatever time they need to 
process the information presented during CBI. 

Guideline » 3: 

Student control of pacing within CBI is 
strongly advised. (Ironically, this strategy may 
prove difficult for local implementers as they 
attempt to manage students completing 
instructional units at widely varying times.) 

Feedback Mechanisms: 

Appropriate instructional interventions or 
feedback must be available at the appropriate 
time to ensure optimum learning (Skinner, 
1974). Response strategies generally employed 
in CBI include feedback regarding results, 
branching to alternative instructional 
sequences, repetition of previous instruction, 
and remediation of discrete conceptual or 
procedural errors (Bangert -Drowns, Kulik, 
Kulik, & Morgan, 1991). Appropriate feedback 
confirms the learner's expectancy, directs 
attention to relevant factors, and stimulates 
recall of relevant skills and knowledge (Gagne, 
1977). Feedback for motivation is one of the 
few aspects of teaching recommended by all 



instructional theories (Heywood. 1986). 
however research concerning learner control 
of feedback has yielded mixed results. 

Belland, Taylor, Canelos, Dwyer, and Baker 
(1985) found that the higher performing 
students were more likely to select elaborative 
feedback than the lower performing students 
who needed the elaborative feedback. Sales 
and Williams (1988) found that learner control 
of feedback selection resulted overall in 
students asking for knowledge of correct 
response or elaborate feedback more 
frequently than simple knowledge of results. 
Their findings conflict with other research 
(Carrier, 1984) that suggests that students will 
choose to see less material when given an 
opportunity to decide. Sales and Williams 
(1988) did not find differential effects for type 
of feedback on achievement or instructional 
time. On the other hand. Schloss. Wisnlewski, 
and Cartwrlght (1988) reported that students 
who received feedback about their cumulative 
performance in conjunction with learner 
control outperformed students under learner 
control alone. 

Guideline # 4: 

The type of feedback options (right or wrong, 
correct response, elaboration) should be based 
upon instructional requirements. If more 
elaborative feedback than knowledge of results 
Is appropriate, allowing student control of 
access to the feedback is generally advised. 
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Learner Control with Advisement: 

The majority of the studies described above 
used variants of learner control unaided by 
advisement. There Is evidence that learner 
control supported by some forms of assistance 
(e.g., guidance on recommended sequences of 
instruction or advisement as to the current 
state of achievement) is more effective than 
unaided learner control (Tennyson, 
Christensen, & Park, 1984). Steinberg (1989) 
concluded that "Students do not perform as 
well under (unaided] learner control as under 
adaptive computer control" (p. 1 18). 

Adaptive (shared) control strategies allow some 
learner control during instruction and impose 
program control only if it is determined that 
the learner is not achieving the desired 
outcomes. Sales and William} (1988) employed 
adaptive control in their study. After an initial 
period of learner control, subjects in the 
adaptive treatment were required to achieve or 
exceed a predetermined success rate on the 
practice items, and if performance dropped 
below the established criterion, the program 
assumed control of feedback selection. 

Research on adaptive/advisement control has 
not been uniformly positive. Adaptive control 
strategies require greater instructional time to 
complete, with no associated gain in 
achievement in a study conducted by Goetzfried 
and Hannafln (1985). On the other hand, 
Tennyson and Buttrey (1980) found that higher 
performance occurred when students were 
provided with coaching or advisement as to 
what instructional resources to select. Ross 
and Rakow (1981), Carrier, Davidson, 
Williams, and Kalweit (1986), and Kinzie, 
Sullivan, and Berdel (1988) also reported 
positive effects for adaptive control over 
unaided learner control. 

Guideline # 5: 

Learner control with advisement is highly 
recommended over unaided learner control. 
The nature of the advisement will be dependent 
upon the sophistication of the performance 
measures incorporated into the CBI as well as 
knowledge of learner error patterns. 

Motivation Research 

Motivation is one of the most extensively 
researched constructs in psychology and 
education, and hence a complete review of 
motivation theory and research in this paper is 
impossible. Ten clusters of motivation theory 
and research are reviewed below accompanied 
by application guidelines for designers and 
users of CBI. 



Attribution Theory: 

Proponents of attribution theory assume that 
the search for understanding of the reasons for 
success or failure is a mainspring of human 
motivation (Weiner, 1972). Attributional 
responses Involve attributing causes to success 
or failure outcomes. Locus of control is a major 
factor related to attribution theory. Students 
with an internal locus of control think of 
themselves as being responsible for their 
behavior whereas students with an external 
locus of control attribute their behavior to luck 
or other circumstances beyond their control. 
Students with an internal locus of control tend 
to have higher need for achievement and are 
more persistent in a learning context. Students 
with an external locus of control are often 
described as having "learned helplessness" 
with respect to areas of study. 

Guideline # 6: 

The success of CBI can be maximized by 
providing students with instructions and 
feedback that would encourage them to make 
Internal attributions about their ability and 
effort related to their success. This argues for 
some variant of learner control with 
advisement whereby students are given advice 
about their instructional progress and needs, 
but the actual selection of instructional 
strategies would be left up to them, 

Self-Esteem Theory: 

The basic tenet of the self-esteem theory of 
motivation is that a student's behavior is 
largely a function of his or her self-concept 
(Shavelson, Hubner, & Stanton, 1976). For 
example, if a student thinks of him/herself as a 
"class clown," the student will act like a clown. 
Self-esteem Is a very complex construct, and 
recent evidence Indicates that it may not have 
as strong a link to performance in all subjects 
as was once thought. For example, in recent 
international comparisons of achievement in 
mathematics and science, Korean students 
scored the highest and yet rated themselves 
near the bottom in terms of self-esteem. U.S. 
students scored the lowest, but rated 
themselves the highest in terms of self-esteem! 

Guideline* 7: 

CBI should be designed to enhance the self- 
esteem of students through positive 
affirmations. However, repetitive, in iocuous 
affirmations such as "That's great" or 
"Correct" will be ignored by students after a 
short time. Contextual, individualized, 
creative affirmations are recommended 
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Curiosity Theory: 

Curiosity tiieory maintains that curiosity, as a 
state of mind, encourages exploratory behavior 
and hence learning (Vidler, 1977). Moderately 
unfamiliar stimuli are said to promote 
curiosity whereas completely novel stimuli 
promote fear and completely familiar stimuli 
promote boredom. Curiosity seems to be 
closely related to an individual's creativity. 
Both are very difficult to measure. 

Guideline # 8: 

The navigation procedures and basic structure 
of CBl should be consistent, but moderate 
variations in style of presentation and content 
will stimulate beneficial curiosity. 

Western Electric Productivity Studies: 

A series of research studies were conducted at 
the Hawthorne works of the Western Electric 
Company near Chicago, Illinois in the late 
1920's (Pennock, 1929). These studies found 
that virtually any change in the working 
environment (both positive and negative) led to 
increased productivity. The primary 
explanation for the results is that productivity 
increased simply because the workers were 
made to feel special while they were being 
studied. This became known as the 
"Hawthorne Effect." 

Guideline # 9: 

Whenever a new technology such as CBl is 
introduced, a certain amount of "Hawthorne 
Effect" is inevitable. However, if the CBl is 
well-designed and effectively implemented, the 
perceived and real value of it wti.1 continue long 
after the novelty has worn off. 

Theory X and Theory Y: 

Researchers from the Massachusetts Institute 
of Technology described two theories of 
management that have major Implications for 
motivation, viz.. Theory X and Theory Y 
(Carlisle & Murphy, 1986): 

Theory X maintains that: 

1) Most people, by nature, don't like to work. 

2) Most people lack ambition and need a club 
over their heads to make them work. 

3) Most people prefer to be told what to do. 

4) Most people resist change. 

5) Most people are gullible and not overly 
Intelligent. 

Theory Y maintains thai: 

1) People do not dislike work but may actively 
seek it. 

2) People do not need the authoritarian type of 
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leadership but prefer a participative kind of 
management. 

3) People prefer setting their own goals rather 
than have someone else set them. 

4) People do not shirk responsibility but 
rather seek it. 

Guideline #10: 

// the content and context allow, CBl should be 
designed to enable students to set their own 
goals and develop at their own rate. 

Mallow's Hierarchy of Needs: 

Maslow (1954) developed a well-known 
hierarchy of needs that has five levels: 1) 
physiological needs, 2) safety and security 
needs, 3) social needs, 4) esteem and self-respect 
needs, and 5) self-realization needs. One basic 
premise of his theory is that as lower order 
needs are satisfied, other needs become more 
apparent. 

Guideline ill: 

The lower level needs of students must be 
satisfied if they are expected to be motivated by 
higher order needs. The environment in which 
CBl is implemented must be comfortable, non- 
threatening, and supportive. 

Expectancy Theory : 

Proponents of expectancy theory believe that 
motivation is directly correlated with how 
much we want something and the probability 
that certain activities and actions will lead to 
obtaining it (Spence, 1960). A derivation of 
this theory called "mental imagery" or 
Visualization" has been used with success in 
athletics and other fields* Some research 
supports the notion that the expectancies of 
trainers can affect the behaviors of learners. If 
a trainer believes that his or her students are 
bright, active, and articulate, they may well 
behave that way. Unfortunately, the reverse 
may also be true. 

Guideline * 12: 

Implementers of CBl should have positive 
expectations with respect to their students. CBl 
should be designed to encourage students to 
expect success. 

Human Resources Development (HRD): 

The HRD perspective maintains that 
motivation is 100% internal (Laird, 1978). 
HRD specialists believe that behavior Is 
influenced by motivation and external 
pressures. They maintain that the only way to 
get someone to do something (e.g., learn) is to 
make him or her want to do it. HRD specialists 
advocate using a positive approach to getting 
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someone to want to do something. A positive 
approach can involve anything that leads to an 
Increase In jatisfactlon. HRD specialists 
recognize that people may be motivated by fear 
or danger, but they say such methods should be 
used only as a last resort. 

Guideline § 13: 

CBI should be designed to take advantage of 
whatever is known about the students 9 internal 
motives so that their level of satisfaction is 
enhanced 

Malone's Theory of Intrinsically Motivating 
Computer-Based Instruction: 

Malone (1984) has conducted research almtd at 
building a theory of intrinsically motivating 
computer-based Instruction. Much of his 
research has revolved around discovering what 
makes computer-based games so captivating. 
According to Malone, for CBI to be 
intrinsically motivating, it must have 
characteristics such as 1) challenge, 2) fantasy, 
and 3) curiosity. To be challenging, CBI should 
have a clear goal whose outcome is uncertain. 
There should be variable levels of difficulty 
and multiple levels of sub-goals. Using fantasy 
in CBI involves building in game -like 
structures that promote competition with the 
machine and/or other students. To stimulate 
curiosity, Malone advocates the use of 
graphics, video, animation, and/or music in 
CBI. 

Guideline #14: 

Trainers may want to incorporate some of the 
features of computer-based games into CBI, but 
only {f the nature of the content and context 
indicates that it is appropriate. 

Keller's ARCS Model of Motivation: 

Keller (1987a, 1987b) developed a practical 
model for Incorporating motivational aspects 
into instruction. Keller defined four major 
categories of motivation in instruction, 
Attention, Relevance, Confidence, and 
Satisfaction. Hence, the model is known as the 
ARCS model. He also defined twelve design 
activities to guide the design of instruction in 
ways that enhance the intrinsic and extrinsic 
motivation of the learners. 

Guideline* 15: 

CBI designers should include Keller's design 
activities into their overall instructional 
development model to maximize the 
motivational aspects of their programs. 

Conclusion 

Despite the lack of a strong theoretical and 



research basis for the interrelationships 
between learner control and motivation, we 
have derived fifteen guidelines for design and 
implementation of CBI with reference to these 
two critical factors. A final caveat is 
warranted in that much of the learner control 
research cited In this paper is flawed in terms 
of sample sizes, treatment durations, subject 
selection, etc. (see Reeves, in press, for a 
detailed critique). Nonetheless, the theoretical 
and research basis for the guidelines we have 
included seems sufficient for their serious 
consideration by authors and users of CBI. 
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ABSTRACT 

Is personality type a factor In whether teachers 
volunteer for technology training? Does it 
influence their adoption of technology? The 
Myers Brlggs Type Indicator (MBTT) was given 
to 240 technology training volunteers. After a 
year long training cycle, they were rated on 
level of adoption. The technology volunteers 
differed significantly from the general 
population at the .01 level on 4 of 5 
comparisons of personality type by 
participants. Statistical significance was 
attained (p_< .05) in 13 of 29 comparisons of 
personality type by subject area. The typical 
personality type of the technology training 
volunteers is Introvert iNtuitive Thinking 
Judging (INTJ). The different personality type 
does not appear to influence their rated level of 
technology adoption. The profile for this 
personality type describes an independent 
person who actively embraces an innovation 
because it represents a problem to be solved. 
They may not be as successful at getting others 
to adopt the change. Long term change 
strategies must offer incentives that appeal to 
other personality types as well. 

Some teachers volunteer for technology 
training because they are interested. Some 
volunteers are already active users and Just 
want to learn more. Seme teachers are 
volunteered because they seem like the type. 
Others are volunteered because they are 
available. Whatever their reason for attending 
training, many of these teachers will become 
successful adopters of technology and others 
will not . 

Is personality type one of the factors that 
influence whether teachers volunteer for 
technology training? Does it influence how 
well they adopt technology? Can school 
districts use personality type as a predictor of 
successful training participation and 
innovation adoption? 

Personality Type 

Jung's theory of psychological type describes 
personality factors as a key to an individual's 
motivation and academic or job performance 
(Lawrence, 1984). His theory considers the 



conscious use of alternate ways to receive and 
make decisions about information from the 
environment (McCaulley, 1990). 

Myers and Brlggs expanded the theory into the 
Myers Brlggs Type Indicator (MBTI) (Myers. 
1978) which has been used for over 40 years by 
counselors, educators and organizations to 
identify underlying personality strengths and 
weaknesses. The MBTI items are concerned 
with four bi-polar preferences on which the 
user chooses a relative preference. 

Extraversion-Introversion attitudes represent 
ways of relating to the world and people. 
Extraverts (E) are continuously alert to people 
and events outside themselves, have a wide 
variety of interests, and take an active, trial 
and error approach to life. Introverts (I) direct 
their energy inward, pursue fewer interests but 
explore them more deeply, and take a reflective 
approach to life. 

Senslng-iNtuitlng perceptions represent ways 
of taking In information. Being practical and 
fact oriented. Sensors (S) attend to the literal 
meanings of concrete experiences and learn 
best by moving step by step through a new 
experience. INtuitors (N) perceive through 
associations and memory, attend to implicit 
meanings, and show more concern with 
Insights and opportunities in the future than 
with the reality of the present. They learn in 
skips and Jumps, looking for patterns and 
using imagination. 

Thinking-Feeling judgments represent ways of 
making decisions. Thinking (T) types make 
decisions through a logical, impersonal, 
analytical and organized analysis of causes 
and effects. The decisions of feeling (F) types 
consider harmony, the feelings of others, and 
the relative importance or value of competing 
alternatives. In our culture, sex differences 
between thinking and feeling types are 
pronounced with females favoring feeling and 
males thinking. 

Judging-Perceiving represents lifestyle 
orientation and is reflected in work habits. 
Judging (J) people like to move quickly toward 
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decisions and enjoy organizing, planning and 
structuring. Perceptive (P) people enjoy being 
curious and open to changes, preferring to keep 
options open in case something better turns up. 

MBTI results provide a profile of an 
individual's personality type. One dimension 
of each subscale is identified as dominant and 
an individual Is described as one of 16 types, ie, 
INFP, ESTJ, ENTJ, etc. 

The sixteen types are not evenly distributed in 
the MBTI data base population (Lawrence, 
1984). Extraverts make up 70% while 
Introverts represent 30%. Sensors are 70% 
while iNtuitors are 30%. Thinking and Feeling 
have different distributions for male and 
female: for Thinking, 60% are male and 40% 
female; for Feeling, males are 40% and females 
60%. Judging is 55% and Perceiving 45%. The 
predominant pattern in our American culture 
is ESTJ for men and ESFJ for women. 

Type and Teachers 

Individuals choose careers because their 
personality characteristics fit the 
characteristics of the career (Holland, 1373). 
All 16 types are represented among teachers. 
Extraverts (E) and Introverts (I) are about 
equally represented at all levels of teaching. 
The proportion of iNtuitive (N) types increases 
as grade level rises with Sensing predominant 
through high school. Feeling (F) types are in 
the majority through high school with 
Thinking (T) types outnumbering Feeling types 
at the college level. At all levels, Judging (J) 
types outnumber Perceptive (P) types 
(Lawrence, 1984, p. 21). The predominant K - 12 
teacher type is ESFJ. Comwell, Hegelson and 
Wachowiak (1987) reviewed MBTI validity 
research in educational settings and 
substantiated the preponderance of the SF 
types in teaching at elementary and middle 
school levels. 

Kolb's work on the Learning Style Inventory 
(LSI) (Kolb, 1984) offers evidence on teachers' 
type by content areas. Kolb cites 
correspondences between the MBTI's 
Introversion and the LSI's reflective 
observation, and between extraversion and 
active experimentation. He says concrete 
experience is clearly associated with both the 
sensing approach to perception and the feeling 
approach to judging. Abstract 
conceptualization, on the other hand, is related 
to the* intuition approach to perceiving and the 
thinking approach to Judging (Kolb, 1984, pp. 
79. 80). 
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Elementary and secondary educators cluster in 
the concrete experience quadrant. However, 
differences are found in content area 
subgroups. In a review of 36 academic 
specialities, the following classifications were 
made: vocational, special, secondary, and 
educational administration were in the 
concrete experience quadrant; math and 
science teachers were in the reflective 
observation quadrant; and social science and 
language teachers were in the active 
experimentation quadrant. This translates 
into MBTI terms by identifying more math and 
science teachers as Introvert and more Social 
Studies and Language teachers as Extravert. 
Vocational and special teachers are more 
Sensing. 

Technology and Type 

Students who are successful at using computers 
and other types of technology for learning 
show a different personality pattern from the 
typical population. Seven studies found 
significant relationships of personality type 
with use of various styles of technology based 
instruction. No studies of technology teachers' 
types were identified. 

Five studies used computer assisted instruction 
(CAT). Carter (1985) studied 132 Alrforce 
Institute of Technology participants who were 
more ISTJ with Si' most frequent and NF least 
frequent. ENP types were disproportionately 
likely to drop out in Hoffman, Waters and 
Berry's (1981) study of 100 men and 55 women 
in a military training setting that used CAI. 
Conversely the ISJ types did not drop out as 
often. S types tended to complete CAI programs 
sooner than N types. 

Gaston (1973) found that senior dental students 
at Ohio State who had favorable attitudes 
toward CAI tended to be S and J. Hopmeier 
(1981) reviewed other studies and concluded 
that I types are better suited to CAI than E 
types. Kern and Matta (1988) studied 90 
undergraduate business students who used self 
paced instruction to learn Lotus 1-2-3. S types 
were more successful that N types. 

Two studies investigated individualized 
learning with other types of technology. 
Atman (1989) identified type as an inadvertant 
contributor to success in distant learning 
courses. Students with higher goal oriented 
behavior profiles were ENTJ. Early (1933) 
found that of 2300 medical students at Ohio 
State, more N chose individualized over 
traditional instruction. 
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Combining these findings, students who had 
success and/or satisfaction using technology 
based Instruction were more often Introvert 
Sensing Thinking and Judging (ISTJ). Myers 
(1978) profiles the ISTJ type as liking things 
kept factual, clearly stated and not too 
complex. They are organized, accurate, 
systematic, hard working and patient with 
detail and routine. 

This study aimed to identify differences in 
personality type of teachers who volunteered 
for and participated in technology training and 
who subsequently were successful In adopting 
technology. Two hypotheses were tested in this 
study 

1. Technology training volunteers have the 
same personality type as the typical teacher. 

2. Personality type does not afTect adoption of 
technology. 

METHODOLOGY 

As this study took place, the New Kid Graduates 
Project was in the final phase of a five year 
effort to install technology in all schools in the 
Jefferson County Public School District. All 24 
high schools were equipped with thirty-two 
unit networked Macintosh labs. Additional 
resources included integrated applications 
software, HyperCard, a desk top publishing 
station with digital image scanner, and a 
multimedia station with interactive videodisc 
and CD ROM. 

A team of 12 teachers in each high school 
received 60 hours of training. A total of 
approximately 300 teachers participated 
during the two, year long phases. Some of the 
participants volunteered when the opportunity 
was announced, others were enlisted by their 
peers, and some were volunteered by their 
principal or department head. The training 
was conducted by the school district's team of 
technology support staff. Integrating 
technology as a tool in content areas was the 
primary focus. 

The participants were expected to use the 
technology to teach their students. In this way, 
they would model for other teachers how to use 
technology for Instruction. They were also 
expected to share their expertise with non- 
trained teachers by demonstrating, coaching, 
and providing inservice sessions. 

The MBTI-Form G, Self Scorable version was 
administered during the early weeks of 
training. Of the 300 participants, 240 (80%) 
completed It in usable form. 



During the year, the districts technology 
support staff provided on-site assistance and 
follow up. At the end of the year, they rated 
each participant on degree of adoption of 
technology. Adoption levels were defined as 
follows: high level - enthusiastically and 
consistently use all or most of the technology 
with students assigned to their courses, and are 
willing to share with other people; medium 
level - use some technology for personal 
purposes and some use with students; low level 
- have not brought students to the lab nor 
developed an Independent personal level of use. 

RESULTS 

The participants were teachers of English, 
Science, Math, Social Studies and Other (home 
ec, business, vocational, special ed, art). Thirty 
percent of the participants teach English. This 
large number might be expected since the 
project focused on writing In content areas. 
Other subjects were less well represented. The 
Other category made up twenty-eight percent of 
the participants. The sex distribution of the 
participants in the study was 69% female and 
31% male. 

Table 1 shows the distribution of subject area 
by sex for the 240 subjects. A chi-square test 
revealed no significant relationship between 
the variables, x 2 (4, N a 240) = 8.60, n > .05. 
Thus, there was no evidence that the 
proportion of females to males differed by 
subject area* 

Table 2 shows the proportion of subjects in the 
various categories of the MBTI. A total of five 
significance tests were performed on these 
data. Each time, a chi-square goodness-of-flt 
test was performed using proportions taken 
from the general population as expected 
proportions and using the obtained 
proportions from the subjects. If proportions 
of study subjects closely matched that of the 
general population, then the null hypothesis 
was retained. Otherwise, the null hypothesis 
was rejected. 

In the general population, .70 of persons are 
Extraverts and .30 Introverts. For the subjects 
of this study, .47 were Extraverts and .53 were 
Introverts, a significant deviation, x 2 (1, H = 
240) = 60.02, sl < .01. Similarly, .48 of the 
subjects were Sensing and .52 were INtuitlng, a 
significant departure from the proportions of 
.70 and .30, 
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Technology Training Participants by Subject Area and Sex 





Science 


Math 


English 


Social 


Other Total 


Female 


0.10 


0.08 


0.20 


0.06 


0.22 0.69 


Male 


0.06 


0.07 


0.09 


0.01 


0.06 0.31 


Total 


0.18 


0.15 


0.30 


0.08 


0.28 1.00 



Table 1 



Personality Type by Participants 



MBT1 


General 
population 


Number of Proportion 
participants of participants 


Chi square 


a 


Extravert 


0.70 


113 


0.47 


60.02 


0.00 


Introvert 


0.30 


127 


0.53 






Sensing 


0.70 


114 


0.48 


57.86 


0.00 


IMtuiting 


0.30 


126 


0.52 






Female 












Thinking 


0.40 


97 


0.59 


24.27 


0.00 


Feeing 


0.60 


68 


0.41 






KAale 












Thinking 


0.60 


51 


0.68 


2.00 


0.16 


Feeing 


0.40 


24 


0.32 






Judging 


0.55 


159 


0.68 


12.27 


0.00 


Perceiving 


0.45 


81 


0.32 







Table 2 

x 2 (1. N = 240) = 57 .86) fi < .01. 

In the general population, .40 of females are 
Thinking and .60 are Feeling. However, .59 of 
the females in this study were Thinking and .41 
were Feeling, x 2 (1,M= 165) = 24.27, a < .01. In 
contrast, the males in this study did not 
significantly deviate from expectations. The 
proportions of .68 Thinking and .32 Feeling 
were close to the proportions .60 and .40, x 2 (1, 
N=75) = 2.00,p>.05. 

Finally, the data was analyzed on the MBTI 
categories of Judging and Perceiving. It was 
found that .68 of the subjects were Judging and 



.32 were Perceiving, a significant departure 
from the general population proportions of .55 
and .45, x 2 (1, M - 240) = 12.27, p < .01. 

In summary, the data revealed that, compared 
to the general population, participants in this 
study were more likely to be Introverted rather 
than Extraverted, iNtuiting rather than 
Sensing, and Judging rather than Perceiving. 
In contrast to the general population, females 
in the study were more likely to be Thinking 
than Feeling. However the ratio of Thinking to 
Feeling males in this study was not different 
than general data for males. 
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Personality Type by Subject Area 





Science 


MBTI 


Prop 


x2 


Extravert 


0.50 


8.38 


Introvert 

It 111 W f Vll 


0.50 




densina 


0.55 


5.00 


iNtuitina 


0.45 




Female Thinkina 


0.68 


8.17 


Female Feelino 


0.32 




Male Thinkina 

If IMIV 1 ■ III 11*11 IM 
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Prop 
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Extravert 


0.45 


5.95 
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Engfsh 
Prop x2 
0.49 15.68 
0.51 

0.44 22.39 
0.56 

0.68 16.33 
0.32 

0.64 0.12 
0.36 

0.67 3.96 
0.33 



ft 

0.00 

0.02 
0.00 
0.45 
0.40 

ft 

0.01 

0.14 
0.18 

1.00 



Math 
Prop 
0.50 
0.50 
0.42 
0.58 
0.52 
0.48 
0.67 
0.33 
0.81 



x2 
6.86 

13.76 

1.34 

0.28 

9.50 
0.19 



ft 

0.01 

0.00 
0.25 
0.59 
0.00 

ft 

0.00 
0.00 
0.18 
0.74 
0.11 



Total 
Prop 

0.47 

0.53 
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0.41 

0.68 

0.32 

0.66 

0.34 



x2 
0.91 

2.62 

5.10 

2.65 

4.97 



ft 

0.00 

0.00 
0.00 
0.73 
0.05 

ft 

0.92 

0.62 
0.26 
0.16 
0.29 



Other 

Prop x2 
0.43 24.23 
0.57 

0.53 9.42 
0.47 

0.49 1.81 
0.51 

0.64 0.11 
0.36 

0.61 2.59 
0.39 




The technology volunteers may be seen as 
sharply different from the general and teacher 
populations with only the Judging variable in 
common. They are more like technology using 
groups described in other studies on Introvert, 
Thinking and Judging variables. 

Study population INTJ 
General population - women ESFJ 
General population - men ESTJ 
Teacher population ESFJ 

As a follow up to the analysis of data in Table 2, 
further analysis was performed, taking into 
account subject area. Table 3 shows the 
proportion of subjects in each MBTI type by 
subject area of the teacher. Separate chi- 
square goodness of fit tests were performed for 
each subject area (Science, Math, etc.) using the 
general population figures of .70 Extravert and 
.30 Introverts. For each subject area, the null 
hypothesis was rejected (p. < .01). Within each 
subject area, the proportion of Introverts 
exceeded expectations. 

The data for Sensing and iNtuiting were 
analyzed in a similar manner. With the 
exception of Social Studies, a significantly 
greater number of teachers (ft < .05) were 
iNtuiting. 

The proportion of Thinking and Feeling 
females was compared, again using separate 
chi-square goodness of fit tests for each subject 
area and the expected female proportions of .40 
Thinking and .60 Feeling. The null hypothesis 
was rejected (B < .01) for teachers in the areas of 
Science and English. In both of those subject 
areas, there was a greater than expected 
proportion of Thinking teachers than Feeling. 
Using the same comparison and the expected 
male proportions of .60 Thinking and .40 
Feeling, the null hypothesis was retained for 
male teachers In all subject areas. The ratio of 
Thinking to Feeling male teachers paralleled 
that expected in the general U.S. male 
population. 

Finally, separate chi-square goodness of fit 
tests were performed for each subject area using 
the general population figures of .55 Judging 
and .45 Perceiving. The null hypothesis was 
rejected (ft < .05) in Math and English because 
the proportion of Judging teachers was higher 
than expected. 

The analyses of MBTI types within each subject 
area were generally consistent with analysis of 
the data combined across subject areas that 
was reported for Table 2. Introverts were more 
common than expected for all areas except 



Social Studies. A greater than expected number 
of female Science and English teachers were in 
the Thinking rather than Feeling category. As 
was found with the general male population, 
most of the male teachers in this study were in 
the Thinking category: this was true for all 
subject areas. Finally, a greater than expected 
proportion of Math teachers and English 
teachers were in the Judging rather than 
Perceiving category. 

Table 4 shows data on the level of adoption of 
technology after training. Of the 240 
participants, 152 (64%) were rated as "High" or 
"Medium" in their 

level of adoption. This is clear evidence of the 
efficacy of training. Thirteen percent dropped 
out before completing the project 

In addition, the effects of MBTI category on 
adoption level were studied. Subjects rated 
high, medium and low on adoption level were 
coded 3, 2, and 1 respectively. Then an 
independent t test was performed with 
adoption level rating as the dependent variable 
and MBTI category as the independent 
variable. Table 5 shows no significant 
differences. The average adoption level was not 
significantly different for Extravert compared 
to Introvert, Sensing to INtuiting, Thinking to 
Feeling, or Judging to Perceiving. One 
interpretation of these 



Adoption Level After Training 




Level 


N 


C 


High 


76 


0.32 


Medium 


76 


0.32 


Low 


57 


0.24 


Drop Out 


31 


0.13 


Total 


240 




Table 4 







results is that training was uniformly 
successful across all MBTI types. Type did not 
appear to influence levels of technology 
adoption in this study. 
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Adoption Level by Type 



MBTI 


Mean 


t 


Extravert 


2.14 


1.00 


Introvert 


2.03 




Sensing 


2.02 


•1.22 


iNtuHing 


2.15 




Female Thinking 


2.20 


0.60 


Female Feeling 


2.13 




Male Thinking 


1.87 


-0.33 


Male Feeling 


1.94 




Judging 


2.09 


-0.12 


Perceiving 


2.10 




Table 5 







DISCUSSION 

The results of this study indicate that the 
teachers who volunteered for technology 
tra'ning are a different type. However, there 
were no significant differences among the 
variations of personality as to how well they 
adopt technology. The training provided by the 
school district coupled with their natural 
inclinations led approximately two thirds of 
them to use technology for themselves and/or 
their students. About one third also shared it 
with other teachers. 

Myers' (1978) profile of the INTJ type paints 
them as independent Innovators in thought 
and action who are stimulated by problems. 
They will spend any amount of time and effort 
to see their inspirations worked out in practice 
and accepted by the rest of the world. They 
may, however, fail to actively check the 
viewpoints and feelings of others, resulting in 
bitter rejection. 

This type of person would actively embrace an 
innovation but might not be as active or 
successful in getting others to do so. If long 
term change Is the goal, school districts need to 
be sure there are incentives that cause other 
personality types to become involved. 

While the findings of this study are revealing, 
additional questions need to be addressed. 
Have the teachers continued and increased 
their use of 



df a 
200.00 0.32 



207.00 0.22 



142.00 0.55 



62.00 0.74 



207.00 0.91 



technology? In what ways are they using it 
with students? Are they effective In helping 
other teachers in the school adopt technology? 
What overall impact has technology use had on 
the faculty and the climate of the school? 
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COMPUTER-BASED COGNITIVE APPRENTICESHIPS: AN 
EXAMPLE PROM WEATHER FORECASTING 



Brent Wilson, University of Colorado 
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Sherwood Wang, University of Colorado 



The purpose of this paper is (1) to outline 
strategies for turning computer-based 
instruction into "cognitive apprenticeships", 
and (2) to illustrate a cognitive-apprenticeship 
approach to teaching weather forecasting 
Icnowledge and skills through the design of an 
nteractive multimedia computer-based 
learning (CBL) module. We report preliminary 
evaluation data on an interactive multimedia 
CBL module titled Boundary Detection and 
Convection Initiation (CI). Convection 
Initiation, a module that includes laserdisc 
audio and video to simulate a forecaster 
workstation and provide problem- solving 
practice to forecasters. The cognitive 
apprenticeship model provides the conceptual 
foundation for a continuing series of IVD 
modules being developed by COMET, a 
cooperative of federal agencies and universities 
involved in meteorological training. 

Background 

The Cooperative Program for Operational 
Meteorology, Education and Training (COMET) 
is an organization under the University 
Corporation for Atmospheric Research 

(UCAR) 1 which has recently embarked on a 
comprehensive interactive multimedia CBL 
program for the operational meterological 

organizations of the United States 2 (Bonner 
and Spangler, 1991 ). 

The focus of this paper is on one of the first two 
CBL modules contained in a comprehensive 
Forecaster's Multimedia Library. These two 
modules were recently field tested. In addition 
to these modules, the next two modules are 
under development. Heckman (1991) outlines 
the instructional design stratgies and 
additonal details on these first two modules. 

The Cognitive Apprenticeship Model 

Advances in cognitive psychology are 
continuing to influence the design and 
implementation of teaching technologies. In 
particular over the last several years, 
researchers have compared the often 
remarkable learning accomplishments in 
natural settings (home, work, and play) with 



the disappointing learning outcomes accruing 
from formal school settings (Rogoff & Lave, 
1984; Rogoff, 1990). What is needed, according 
to Rtsnick (1989), is more "bridging" devices or 
technologies that bring realistic contexts and 
problems into the classroom. The Vanderbilt 
group (Bransford, Sherwood, Hasselbring, 
Kinzer, & Williams, 1990) have demonstrated 
the use of videodisc sequences-e.g., Sherlock 
Holmes or Indiana Jones- to establish 
meaningful "macrocontexts" from which 
problems in a variety of subject areas can be 
explored. 

Collins, Brown, and associates (Collins, 
Brown, & Newman, 1989) have developed a 
"cognitive apprenticeship" model that seeks to 
borrow the best features from informal 
learning settings and incorporate them into 
more formalized instruction. The model 
includes four main categories of design 
considerations: content, methods, sequence, 
and sociology. The elements of the model are 
discussed below as they relate to the design of 
Convection Initiation, one of the completed 
modules. 

Content. 

Cognitive apprenticeships teach domain or 
textbook knowledge, but they also make 
explicit the strategic knowledge used by experts 
in solving real problems. A key goal for the 
COMET module was to provide authentic 
activities that required the forecaster to use 
situation-specific skills and knowledge to 
perform analyes of meteorological data 
(satellite, Doppler radar, surface observations, 
and others) and to make short-range (0-1 hour) 
forecasts. There are no distinct "concept" 
lessons or "rule" lessons. Instead, a series of 
"practice cases" are presented with 
accompanying followup questions that test the 
learner's prior or gained knowledge. The latter 
is accessable, through learner control, from 
topic specific conceptual models and 
procedural tutorials . Figure 1 shows these 
learning tools in the upper left corner of the 
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Figure It The Workstation Interface for the 
Convection Initiation CBL Module 



user Interface. In short, the domain and 
strategic content Is made available to the 
learner In the Immediate context of solving 
specific forecasting problems. 

Methods. 

Collins et al. (1989) have examined several 
exemplary instructional approaches and 
extracted a set of common teaching methods: 

1. Modelling 

2. Coaching 

3. Scaffolding and fading 

4. Articulation 

5. Reflection 

6. Exploration 

CBL designs are relatively well-suited to the 
first four methods, and a growing number of 
toolkit-style programs and hypermedia 
programs allow for exploration. Reflection- 
comparing one's own problem-solving 
processes with others' Is possible but somewhat 
difficult with CBL. A continuing challenge for 
CBL design is to develop methods of making 
instruction a natural dialogue rather than a 
forced-choice, linear path. In many respects, 
the COMET modules replicate a sophisticated 
forecasting workstation, where both learner 
and program may Initiate actions toward 



problem solution, presentation, or feedback; 
the overall structure, however, is constrained 
by the lack of natural language processing of 
the computer. 

A key issue faced by the design team was the 
problem of how to present material to learners 
when some of them already knew It in one way 
or another. A partial solution was in the 
learner control allowed by the hypertext 
options described above, but we still were 
concerned that a M rule-exampic-practice M 
template would become tedious to many of our 
learners. This feeling was reinforced when one 
of us attended a 3-day lecture-oriented course 
taught by an expert on the subject to members 
of our target population. Even though the 
expert was a gifted lecturer, students only came 
alive during the three scenario-based 
forecasting exercises. By comparison, the 
hundreds of satellite pictures, movie loops, and 
examples, while informative, did not engage or 
Interest the group in the same fashion. We felt a 
need to present new conceptual and procedural 
information without appearing to be 
"lecturing" our audience. One of the most 
exciting prospects of introducing multimedia 
into the operational forecaster's training arena 
is that it will provide an alternate learning 
experience to all too often traditional 
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techniques described ahove. 

To address this problem, a case-based strategy 
was developed that embedded tutorial 
explanations within actual practice cases. New 
content In the lesson Is not assumed to be new 
to the individual learner; instead, learners are 
encouraged to approach new problems and use 
whatever prior knowledge they may have 
available. Tutorials, Conceptual models, or 
Expert Answers are kept short (1-6 minutes) 
and are always presented in the context of a 
specific topic or practice case in which the 
learner is facing. We believe this strategy 
makes the module seem less "preachy" and 
condescending to experienced weather 
forecasters. 

These ideas are Illustrated in Figure 1. Shown 
in the "image window" is a Doppler radar 
reflectivity image, part of the data sets 
available for this practice case in the topic on 
detecting convergence boundaries. The learner 
has the option of answering the question or 
selecting the Tutorial which explains "how to" 
detect these boundaries using a variety of data. 
In this case, the learner has incorrectly 
identified the location of the boundary; the 
expert has provided a "hint," shown in the 
dialogue box beneath the image window. If the 
learner misses after one additional hint, they 
are forced to select the Expert answer. Here, the 
learner will hear the expert provide the correct 
answer through a series of images with 
associated overlays. 

Sequence. 

Collins and Brown note in their review of 
exemplary teaching models that some 
approaches allow students to explore the 
content In a free way, whereas others 
(particularly in well-defined disciplines such 
as math) require careful selection and 
sequencing of practice cases. Thus the 
sequencing rules may depend on the type of 
content being taught. Sequencing issues were 
obviously important at initial stages of the 
module's planning. Our first Inclination was to 
incorporate a simple-to-complex sequence, 
moving from an examination of the surface 
boundary layer, toward understanding and 
identifying convergence boundaries, and 
concluding with nowcasting exercises 
requiring pinpointing storm initiation in time 
and location. An elaboration-type sequence, 
with successive lessons building complexity 
and reinforcing core concepts, seemed a 
natural sequencing strategy. We were led to re- 
examine our plans, however, and consider a 
looser hypertext sequence similiar to the first 
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module, Workshop on Doppler Radar 

Interpretation. 3 Typically in hypertext, 
learners can browse and proceed through 
material in an unstructured way. This requires 
that materials be designed a way that makes 
them accessible at any point in time within 
instruction, making it harder to build logically 
from simple to complex. Hypertext also allows 
materials to take the form of a database, usable 
for instructional tutorials and practice, but 
also allowing easy access for reference or 
example. Thus we could see competing 
"desirables" at stake in the sequencing 
question: more logical and meaningful 
unfolding of complex content using an 
elaborative model versus greater flexibility 
and reference accessibility using a hypertext 
model. 

Based primarily on the nature of the content 
and the learner population, we have chosen to 
follow a simple-to-complex elaboration 
sequence through careful selection of cases; 
however, within each practice case, a variety of 
hypertext-like options are available to the 
learner, allowing access to Rules, Tutorials, 
Conceptual models, and Expert answers 
(discussed below). Learners are encouraged to 
work through lessons and cases within lessons 
in their given order, but to access the various 
Learning Tools available in varying order as 
needed. Such a hybrid modela mix of well- 
structured problems and hypertext-like 
exploration is probably appropriate for a large 
number of Instructional situations. 

Sociology 

Often neglected by instructional-design 
theories, the social setting of instruction and 
the work environment serve as strong 
constraints and mediators of instructional 
effectiveness. Ideally, learners should be 
initiated into a community and culture, of 
"expert practice." Cooperative and competitive 
factors need to be harnassed and utilized for 
good ends. The intrinsic motivation inherent 
in learning the content should be preserved and 
not preempted by external structures and 
constraints. And learning should occur in 
authentic social contexts where knowledge is 
drawn on to solve meaningful problems. In the 
section below, we describe how social elements 
were built into the Convection Initiation 
module. 

Enhancing Social Design 

The social aspects of the cognitive 
apprenticeship model seem to pose the greatest 
challenge for CBL design. CBL is often 
administered on an Individual basis, with a 
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single learner engaged at a computer 
workstation. Although group work is a 
desirable option, sometimes it is not possible. 
The COMET modules are a case in point. 
Modules are to be completed on the Job, but 
because forecast offices are sometimes staffed 
with only 2 - 3 forecasters on duty at a time, 
learners must complete instruction 
individually while working a shlftdurlng. The 
design challenge is: how can we build in needed 
social reinforcement, convey a sense of 
community in what is essentially an 
individualized learning experience and build 
in dialogue between the learner and experts in 
the our science community? To address the 
cognitive apprenticeship's social design, the 
Convection Initiation module incorporated 
several strategies, including: 

1. The module begins with the learner 
listening in on an informal "roundtable 
discussion" among the three subject matter 
experts used in the module. They are 
discussing how each observation system, 
satellite data, Doppler radar and others, 
will singlularly and collectively improve 
the forecaster's ability to make short-range 
forecasts (nowcasts). This sequence does 
three things: l)it establishes initial 
dialogue with experts who in the past might 
have simply been names on a Journal 
article; 2) it provides an overview to the 
entire module; and 3) it establishes a link 
with the expert's face and voice and the 
numerous "audio only" Expert answef 
sequences provided throughout the 

module 4 . 

2. FollowLig each forecasting practice, one of 
the three experts provides feedback on the 
case in the form of an "expert answer." This 
is a 30 - 90 second explanation of the case. 
Meteorological data on-screen are 
highlighted with cues such as arrows, lines, 
etc. The personalized feedback from the 
actual experts should allow some social 
modeling to occur. The meteorological field 
is fairly small; moreover, most field 
forecasters cannot attend the many annual 
scientific conferences where the "experts" 
typically present papers. In addition, it is 
impossible for the experts to provide 
learning at the sponsor's forecast offices 
(approximately 600 sites). Getting to know 
the experts through the CBL modules should 
help strengthen the sense of community 
within the field. 

3. The module concludes with an intrinsic 
simulation where the forecaster, teamed 



with another forecaster, "Ron," solves a 
series of realistic, time-based forecasting 
problems. The purpose of the simulation at 
the beginning and end is to: (1) establish 
expectancies and activate prior relevant 
knowledge, (2) provide integration of 
knowledge and tie-in back to learners' work 
environments, and (3) to model successful 
forecasters and motivate learners to 
emulate the actors' responses to the 
problem. This third function of the 
simulations compensates in some measure 
for the isolation of individualized 
instruction completed at a computer 
desktop. 

Through these efforts to make the module more 
personalized, the social elements needed to 
support good instruction are artificially 
created. We believe these compensating 
strategies will help improve the reception of 
the modules in the field, and improve the 
attitudes of learner completing the modules on 
an individual basis. 

Results of Field Testing 

Recently, both CBL modules were field tested at 
one NWS foreceast offices and two AWS offices. 
Additional testing will occur at another NWS 
office and a NOC forecast office. Based on 
preliminary analysis of evaluation data, the 
following findings have emerged. 

1. Students prefer the CBL modules over 
traditional course-based instructional 
modes. The dominant reason for this was 
that computer-based learning provided them 
"control" over the experience, they could 
move around the learning material with a 
high degree of freedom, and it was well suited 
for shift workers who usually have to learn 
on an individual basis. 

2. Students are divided In their preference 
between audio-only expert answers and the 
experts' use of chromaboard. While some 
students reported benefiting from actually 
seeing the expert stand up and explain the 
data, others reported that the audio 
explanations tended to be more to the point 
and less distracting. All forecasters, 
however strongly agreed or agreed that 
employing well known experts to provide 
background or experts answers was helpful. 

3. Elements of the cognitive apprenticeship 
were generally well-received by learners. 
These elements include the case-based 
orientation of Instruction, the options for 
accessing a Tutorial or Conceptual Model. 
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Observations obtained by the evaluators 
suggested that forecasters, after 
approximately an hour into CI began 
identifying with the experts, as if the experts 
were "in the computer" and they had 
established some level of dialogue with 
them. The use of "hints" provided in textual 
form by the experts were favorably received 
by the learners; the hints frequently steered 
the forecaster to the correct answer. There 
was little or no negative feeling by learners 
who Incorrectly answered questions; they 
actively sougth (this was true even when they 
correctly answered the question) the correct 
solution using the Expert answer and felt 
that they had "learned something", despite 
the incorrect answer. 

Conclusion 

The Convection Initiation module is designed 
according to a cognitive apprenticeship model. 
The module's goals go beyond content delivery: 
learners should gain confidence in their 
problem-solving skill, and should feel part of a 
community of other expert forecasters. Even 
though our preliminary results lack a 
sufficient data population, the results are very 
encouraging. Additional results will be 
obtained from the second phase of beta testing; 
in addition, a summative evaluation phase is 
planned for each module. Results from these 
additions should ?!«ed more light on the 
effectiveness oi employing a "cognitive 
appreticeship" model. 
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1 UCAR is a consortium of North American 
universities that provide higher education and 
research in atmospheric and ocean sciences. 

2 The primary sponsors of the CBL program is 
the National Weather Service (NWS), the Air 
Weather Service (AWS), and the Naval 
Oceanography Command (NOC). In addition. 
COMET expects to provide modules to a large 
number of universities, foreign weather 
services, industry, and others. 

3 The design team wanted to create a balance 
between a more structured design and a purly 
hypertext strategy. The content and learning 
outcomes of The Doppler Workshop dictated a 
more hypermedia approach. The balance was 
more closely reached in the CI module. 

4 The Doppler Workshop module employs 
a"chromokey" technique where the learner is 
provided audio and visual dialogue with the 
two experts. The design team intentionally 
employed these two different techniques of 
expert dialogue so that a comparison could be 
made. Results will be evaluated later. 
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COMPUTER-BASED LEARNI NG ENVIRONMENTS 
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ABSTRACT 

Telecommunications technology ha3 long been 
associated with the delivery of Instruction. In 
recent years, it has become an integral part of 
many computer-based learning systems. The 
purpose of this paper is to discuss how 
telecommunications technology can be used 
with multimedia systems to enhance single- 
user computer-based learning and describe a 
model for real-time multi-user computer-based 
learning. 

INTRODUCTION 

Historically, telecommunications technology 
has long had an association with the delivery 
of instruction to people of all ages. These people 
were in either a group setting (such as a class- 
room), or in a remote setting (such as an 
Individual home) and the delivery mode was 
audio (radio or tape) or audio /video 
(television); those media continue to be used 
today. 

Initially, the use of radio and television was in 
a passive mode. The user/student were not 
actively involved In the learning experience 
and the instruction, in many cases, was not 
particularly successful because there was no 
live student /teacher interaction. 

The use of television in the learning 
environment provided an early form of a 
multimedia system for learning. It had the 
capability of providing audio, still-frame and 
full-motion video with color to either an 
Individual or group of learners. At the same 
time (and prior to television), motion picture, 
film strip and slides with audio tape provided 
similar kinds of multimedia systems for users. 

These early forms of multimedia systems had 
mixed success as instruments of learning. 
Television was especially threatening to many 
educators who believed that they would soon be 
replaced by a television set In the classroom. 

These early multimedia systems also suffered 
from the fact that the user was still passive in 
the learning experience. While the use of 
multimedia was effective in some cases, the 



user was still not actively involved. In the late 
1970's, the development of the personal 
computer (PC) gave promise of helping solve the 
problem of user Involvement. After all, if a 
computer program could be written to deliver 
instruction in a manner that could be 
controlled by the user, then the problem of 
active Involvement could be solved. Initially, 
the PC had only the capability of presenting 
text with limited graphics on the screen. Most 
of the programs designed for computer-based 
learning were developed by educators who did 
not understand the power and capability of the 
PC. The end result was that many programs 
were somewhat sterile In their impact on the 
user and were really little more effective than 
the textbook itself. At this point In its 
development, it was possible to have 
animation, but no full-motion video associated 
with those computer-based learning programs. 
Also, at this point of development, computer- 
based learning was primarily designed for use 
by a single user on a one-to-one basis. 

Along the same time, there was renewed 
interest in a multimedia technology that had 
its serious beginnings about a decade before 
(the last 60*s and early 70*8). The vldeodlsk had 
the capability of audio, still-frame and full- 
motion video that could be controlled by the 
user. The vldeodlsk soon found a number of 
applications in education and training but was 
expensive to create and operate, however, many 
people felt that the technology had great 
potential. 

It was not long (the early 80 s) until there was 
an integration of the PC and vldeodlsk 
technology so that the PC could control access 
to the vldeodlsk. Since the user could control 
the PC and the vldeodlsk gave the PC still 
frame and full-motion video capabilities, the 
computer-based vldeodlsk system was seen, by 
some educators, as the next best mode of 
Instructional delivery to the classroom teacher 
and, in some cases, surpass some teachers. 

The computer-based vldeodlsk technology was 
(and still is) most appropriate for a single user 
environment. It can be used In a group setting, 
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but there can only be one computer/vldeodlsk 
unit. It Is not possible to use a computer/ 
vldeodlsk system In a group setting (such as a 
networked environment) where multiple users 
can share the same vldeodlsk resource at the 
same time. The reason Is that the vldeodlsk 
contains an analog signal that does not allow It 
to be stored in a workstation s digital memory. 
Hence, only one user can access the vldeodlsk 
at any time, thereby eliminating any 
advantage of a networked environment. 

Technological advances, both In recent years 
and in the near future, in both multimedia 
systems and networking will allow the 
development of a networked (both local and 
wide area) environment for multiple users and 
computer-based learning. In the next several 
sections, we will examine these advances and 
then describe a model for a future computer- 
based learning system. „ 

ADVANCES IN MULTI-MEDIA SYSTEMS 

Current computer/vldeodlsk technology 
clearly demonstrates that the PC can utilize 
and control analog-based multimedia very 
elfectively. Computer-generated graphics and 
vldeodlsk images can be mixed on the same 
screen simultaneously through the use of PC- 
resident graphics overlay hardware and 
software. This technology has limited quality 
audio (FM radio) and video (NTSC television). It 
also has the limitation that videodisks are 
expensive to produce and cannot be changed in 
content once the vldeodlsk Is mastered. Thus, 
the computer/vldeodlsk system has limited 
flexibility. For some applications, the limited 
quality and flexibility are not problems. We are 
now In the age of digitized audio and video. 
This state of affairs has significant 
implications for multimedia systems that are 
controlled by PCs. 

The successful commercial Implementation of 
digital audio, in the form of the compact audio 
disk during the early 1980's paved the way for 
Industry interest in both digitized audio and 
video. While digital audio was first developed 
and played on laser optical systems, digitized 
audio stored on a hard disk soon followed. 
Digital audio products produced much higher 
quality sound reproduction than ever before at 
a cost slightly higher than comparable stereo 
products. 

The appearance of video digitizers and 
scanners made it possible to import still video 
images into the PC's memory where they could 
be combined with digital audio, edited and 
manipulated by the user. The problem was that 



the digitized informal data, especially the 
video images, required a great deal of memory 
for storage and thus significantly limited the 
number of video images that could be stored. 
Compression algorithms were developed and 
these algorithms did help. For example, an 
image that would normally require 10 
kilobytes of storage could be compressed into 1 
kilobyte without significant change to the 
image. The compact disk, read-only memory 
(CD-ROM) technology demonstrated that 
efficient retrieval techniques existed. It was 
now feasible to have a multimedia system that 
could feature audio, text, and a limited number 
of still frame video images. All of this media 
could be stored on a high density disk or a hard 
disk and could be Implemented in a local area 
network (LAN) environment were it could be 
shared by multiple users. But, the multimedia 
system did not have the capability of handling 
full-motion video. 

The statement of the problem and its causes 
relative to full-motion video is relatively 
simple, but its solution is at best difficult. The 
problem involves the following. A video image 
requires a great deal of memory for storage. 
Full-motion video requires a playback speed of 
30 images per second. Without compression, it 
is not possible to transfer the images from 
storage to the screen fast enough to achieve 
motion because of the large amount of data; the 
transfer rate is simply not fast enough. 

The compact Disk Interactive (CD-I) 
technology, yet to see the "light of day", has 
developed a set of compression algorithms io 
solve the problem, but full-motion video is 
achievable on only a fraction (about 1/8) of the 
screen. 

The breakthrough to this problem may come 
from the development work of digital video 
interactive (DVI) technology by Intel- 
Princeton. Prototypes demonstrate short 
segments of full-screen, full-motion video but 
the technology is not yet commerciallv 
available. IBM, in conjunction with Intel- 
Princeton, is testing the technology and hopes 
to Implement it on PS-2s in two or three years. 
Other companies are also studying the full- 
motion video problem but are using other 
techniques such as fractional geometry. 

The DVI technology makes use of dedicated 
chips that compress the video images for 
storage. The compressed images are then 
transferred to a work-station where they are 
decompressed and played back. The transfer 
can be from an external disk to the 
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workstation or from a server on a LAN to a 
workstation. 

ADVANCES IN TELECOMMUNICATIONS 

There have been a number of advances in 
telecommunications technology, particularly 
in the increased capability and reliability of 
transmission media. Voice and data have been 
successfully transmitted for some time. Video 
could also be transmitted successfully through 
the air and over some specialized cables. But, it 
was not feasible to transmit voice, data and 
video over the same media simultaneously. The 
development of coaxial cable and more 
recently, fiber optic technology, have greatly 
improved telecommunications capability. 

The development of high quality, high-speed 
local area networks for communication 
between computers has opened the door to some 
interesting possibilities when combined with 
multimedia systems. 

The first versions of this combination were 
seen in video conferencing. Video conferencing 
allows simultaneous two-way analog video and 
audio between two or more remote sites just as 
though the individuals at each site were in the 
same meeting room or classroom where live 
interaction could take place. Video 
conferencing could occur when sites were 
several miles apart over dedicated 
communications channels. Video conferencing 
has proven to be commercially feasible 
although it is very expensive. Hence, its use has 
been more prevalent in the private industrial 
sector than in learning environments. 
Computers have not typically been a part of a 
video conferencing system. 

The impact of PCs has now been felt in the 
extension of video conferencing technology. 
This new technology, known as multimedia 
teleconferencing, will have a significant 
impact during the 1990's. This technology 
makes use of multimedia technologies to 
provide user, with the ability to effectively 
integrate a wide variety of information 
contained in multiple formats, and from a wide 
variety of sources, into a unified system. The 
drive for the development of teleconferencing 
is the desire to re-create all forms of all 
available information that is possible in a 
face-to-face meeting. For example, a 
conventional business presentation is likely to 
encompass more than the participants seeing, 
hearing and asking questions of the presenter. 
In real-life, handouts are made available, 
proposals are changed and passed around, 
videotapes are viewed, images such as photos 



and graphics are displayed, and computer- 
generated data is presented for review and 
comment. If this type of multimedia 
interaction is available in a typical in-person 
meeting, users contemplate why similar 
capabilities are not also available in existing 
teleconferencing systems. 

In fact they are. A new generation of 
teleconferencing systems and subsystems are 
taking advantage of the processing power 
offered by the PC and the technological 
advances in imaging systems to offer 
multimedia capabilities in electronic 
meetings. 

Such systems are having an impact at all levels 
of teleconferencing, from the traditional 
conference-room setups to the mobile roll- 
about units used for quick meetings. In fact, 
according to some observers, the trend toward 
multimedia capabilities will help bring about 
the long-awaited explosion in desktop, PC- 
based teleconferences, with participants not 
only seeing and hearing each other, but also 
passing and interacting with image-, paper - 
and electronics-based information . (Gold, 
1990 

REAL-TIME MODEL FOR COMPUTER-RASED 
LEARNING 

The scenario described in the previous section 
can be applied to computer-based learning. The 
traditional definition of computer-based 
learning has been that material Is presented to 
a user via a computer, usually a PC, and the 
user reacts to whatever is presented In some 
predetermined manner. There is nothing "live" 
or real-time about the typical computer-based 
learning environment. 

Multi-media teleconferencing could be used to 
enhance computer-based learning in a real- 
time environment. First of all, it would allow 
the presentation to be made in real-time to a 
group of users not necessarily in the same 
physical location. Each user would have a PC 
networked by a LAN (or a wide area network) to 
the Instructor. A class could be conducted with 
the Instructor having two-way audio and video 
communication with each user and each user 
could have the same capability to communicate 
with every other user PC in the group. 
Discussions and problem solving activities 
would be live (real-time). Ideas and proposals 
could be shared with all group members via the 
PC. Instructor notes could be passed to the users 
via the PCs and multimedia presentations 
could be made via the PC. In-class 
examinations could be taken by each student 
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and submitted to the instructor via a digitizing 
tablet or scanner. 

CONCLUSION 

The current model of computer-based learning 
is designed for a single user In a non-real-time 
environment. This environment will be 
enhanced with advances in multimedia 
systems such as DVI. These advances in 
multimedia technology and advances in 
telecommunications technology will allow 
real-time computer-based learning to take 
place in a group environment. 
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Abstract 

PREPnet, the Pennsylvar la Research and 
Economic Network, is one of over 2,300 
regional TCP/IP-based (Transmission Control 
Protocol/ Internet Protocol) networks which 
constitute the Internet. PREPnet's stated 
purposes include: providing electronic 
communications, including significant 
amounts of data and graphic Images, between 
academic institutions and research colleagues 
throughout the Commonwealth of 
Pennsylvania; providing interconnections to 
other networks for worldwide 
communications; and, contributing to the 
economic development of the Commonwealth. 
This overview of PREPnet provides a 
description as to how the network is 
accomplishing these objectives. 

The History of PREPnet 

PREPnet, the Pennsylvania Research and 
Economic Network, was formed in 1987 by a 
consortium of seven public and private 
educational institutions including Carnegie 
Mellon University, Drexcl University, Lehigh 
University, Pennsylvania State University, 
Temple University, University of 
Pennsylvania, and University of Pittsburgh. In 
addition to that provided by the educational 
institutions, significant funding for PREPnet 
was (and still is) provided by Bell of 
Pennsylvania and also by the Commonwealth 
of Pennsylvania. 

Since its founding, PREPnet membership has 
grown from the original seven educational 
institutions to over twenty-five educational 
institutions and a similar number of 
industrial firms. Current negotiations point 
toward an even more rapid increase in 
membership in the future. 

The PREPnet Structure 
Support Personnel 

The entity known as PREPnet is primarily user 
supported. Day to day operations, as well as the 
marketing of PREPnet, are handled via a small 
staff at the PREPnet NIC or Network 
Information Center. PREPnet policies are set 
by the PREPnet Steering Committee, which 
consists of policy representatives from the 
charter members, and, to a lesser extent, by 



policy representatives from the other members 
of PREPnet. As PREPnet expands, discussions 
are underway within the Steering Committee 
as to whether, and how, the Steering 
Committee should be expanded. 

In addition to the policy representative, as well 
as a designated alternate policy representative, 
each PREPnet member also has additional 
support representatives including an 
operational representative, an engineering 
representative, an Information representative, 
and a user services representative. In some 
member institutions a single individual may 
fill more than one of these roles. For instance, 
the operational and engineering representative 
positions are often filled by a single individual 
as are the information and user services 
representative positions. It is the role of the 
operational representative to maintain the 
institution's connection to PREPnet and to 
keep PREPnet operational by aiding in fault 
isolation and diagnosis. The engineering 
representative provides advice to policy 
representatives on issues relating to network 
architecture and design. It is the role of the 
information representative to provide 
Information on PREPnet and PREPnet 
resources to end-users at each institution. 
Finally, it is the role of the user services 
representative to assist end-users in accessing 
PREPnet and the resources available on it. 

Network Layout 

PREPnet is a high-speed fTl - 1.5 million bits 
per second) TCP/ IP-based (Transmission 
Control Protocol/Internet Protocol) network 
with major hubs In Allentown, Philadelphia, 
Pittsburgh, Harrisburg, and State College. A 
Scranton hub Is connected to the Philadelphia 
hub via a 56K line. Future expansion calls for 
addition of a Meadville (northwest 
Pennsylvania) hub in order to remove some of 
the load from the almost full router at the 
Pittsburgh hub; thus, allowing future 
connections to the Pittsburgh hub. 

PREPnet is connected to the Pittsburgh 
Supercomputing Center at the Pittsburgh hub. 
The Pittsburgh Supercomputing Center is also 
connected to NSFNET (National Science 
Foundation Network). Therefore, it is through 
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this gateway that PREPnet connects to 
NSFNET and the rest of the Internet. 

PREPnet Pricing 

Gary Augustson, Penn State's policy 
representative to PREPnet, is quoted as saying 
"the goal of PREPnet is to make connectivity 
affordable for all of our partners in 
Pennsylvania" (Bermar, 1990, p. 98). To this 
end, a rather far-reaching pricing structure has 
been put in place. While a large university or 
non-profit organization pays a membership fee 
of a few thousand dollars a year, a small 
university or non-profit organization only 
pays about one-fifth of that amount. Since the 
Commonwealth of Pennsylvania has made 
major contributions to PREPnet, state - 
affiliated universities are charged an annual 
membership fee of only slightly more than a 
small university. While large commercial 
firms pay a membership fee of about three 
times that of a large university, to facilitate 
economic development small firms only pay 
an annual membership fee similar to that of a 
small university. (Note: A list of the current 
PREPnet pricing structure is available via 
anonymous FTP from the PREPnet Network 
Information Center archives at FTP.PITT.EDU 



In addition to the annual membership fee 
structure, PREPnet members also may choose 
different classes of service in order to keep 
costs down and more closely match the service 
received to their communication needs. 
Options range from high-speed Tl, 56K, or 9.6K 
router-based connections via dedicated lines, 
to slower terminal server-based dial-up 
connections. 

Financial constraints are often found at the 
forefront of PREPnet discussions. PREPnet is 
still working to acquire the critical 
membership mass required to make the 
network self-sufficient. At the same time, 
PREPnet growth must be controlled to 
maintain level loads on the network and at the 
network hubs. The addition of a new hub is a 
major expense which needs to be cost Justified 
by the number of memberships which will be 
using that hub. To maintain controlled growth, 
and to supply the finances needed for this 
growth, the PREPnet pricing structure is under 
constant review. 

PREPnet Growth Potential 
Acceptable Use Policy 

One of the problems in marketing the network 
involves defining what are acceptable uses of 
the network. Having a very restrictive 
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acceptable use policy would have the adverse 
reaction of limiting the number of 
organizations wishing to become members. 
While a very liberal acceptable use policy might 
be sufficient internal to PREPnet, PREPnet 
does connect to NSFNET and the rest of the 
Internet. Having an acceptable use policy 
which differs from the other networks to which 
PREPnet connects would result in information 
flowing through PREPnet which would not be 
allowed into these other networks. It would be 
difficult, if not impossible, to prevent this 
information from leaving PREPnet to these 
other networks. To eliminate acceptable use 
problems with external networks, the 
"PREPnet Acceptable Use Policy and 
Agreement" requires that all traffic exiting 
PREPnet adhere to the "NSFNET Acceptable 
Use Policy." Unfortunately, even the NSFNET 
policy is an Interim policy. 

Academic Expansion 

Discussions are underway to expand PREPnet 
to all academic levels. Recent contracts with 
the Pennsylvania Department of Education 
call for the investigation of the feasibility of 
providing state resources through PREPnet to 
state school districts and Intermediate Units. 
These contracts also include implementing 
access to PREPnet for the Department of 
Education and making the catalog of the State 
Library available to PREPnet users. 
Discussions within the PREPnet Steering 
Committee have focused on making K-12 
connectivity a part of PREPnet's mission. 
However, concerns have also been expressed 
that PREPnet currently cannot handle a flood 
of school districts wishing to be connected. 

At the higher education level, final discussions 
are currently taking place to connect to 
PREPnet the rest of the fourteen institutions 
which make up the State System of Higher 
Education (SSHE). Discussions about adding an 
SSHE representative to the Steering Committee 
have focused on the possibility that SSHE 
schools may have closer ties to school districts 
and could work more closely with selected 
districts to connec ; the districts to PREPnet. 

Negotiations are also underway with a number 
of different library organizations in order to 
make the resources of these organizations 
available on PREPnet. Although these 
organizations already have their own 
networks in place, these networks are 
somewhat less sophisticated than PREPnet. 

Commercial Expansion 

All traffic exiting PREPnet is governed by the 
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"NSFNET Acceptable Use Policy." The stated 
purpose of NSFNET Is to provide support for 
research and education In and among academic 
Institutions. Any activity In direct support of 
this purpose Is considered to be consistent with 
the purposes of NSFNET; hence, acceptable use 
of NSFNET. The main restriction to NSFNET 
use Is that commercial use of the network is 
prohibited. Further, as a member of FARNet, 
the Federation of American Research 
Networks, PREPnet also endorses the FARNet 
guidelines. The FARNet guidelines state that 
traffic between mid-level networks should be 
restricted to research or academic putposes, or 
to direct administrative support of these 
endeavors. 

Adhering to these policies presents PREPnet 
with a rather stringent set of guidelines 
regarding commercial use of the network. 
Commercial firms which conduct research in 
conjunction with academic institutions, 
qualify under these guidelines and may become 
affiliated with PREPnet. On the other hand, 
commercial firms which simply wish to 
electronically communicate with these 
research conducting firms, without working In 
conjunction with an academic institution, do 
not meet these guidelines and hence, traffic 
from these firms may not exit PREPnet. 

Problems arise when commercial firms not 
only conduct research with academic 
institutions, but also market products or 
services to these institutions. While providing 
support of their products to the institutions 
through PREPnet and NSFNET is acceptable, 
marketing new products is not generally 
acceptable on PREPnet, and not at all 
acceptable on NSFNET. Although PREPnet 
encourages commercial affiliation, acceptable 
use guidelines must be observed. In order to 
foster economic development within the State 
of Pennsylvania, the "PREPnet Acceptable Use 
Policy and Agreement" includes as acceptable 
the use of PREPnet "for purposes of technology 
transfer or the economic development of 
Pennsylvania" as long as this traffic remains 
within the confines of PREPnet. Firms which 
are accepted as PREPnet affiliates must sign 
the "PREPnet Acceptable Use Policy and 
Agreement" which also stipulates agreement to 
the "NSFNET Acceptable Use Policy." 

PREPnet Resources 

As PREPnet is part of the Internet, all Internet 
resources are available through PREPnet. In 
addition to Internet resources external to 
PREPnet, numerous resources are available 
directly on PREPnet. The first of these PREPnet 



resources is the PREPnet NIC or Network 
Information Center. Via anonymous FTP (File 
Transfer Protocol) to the PREPnet node 
FTP.PrTT.EDU, users gain access to a vast 
amount of information related not only to 
PREPnet, but also to the entire Internet. 
Available information Includes acceptable use 
policies, lists of PREPnet and Internet 
resources, and even "FYI" documents from the 
National Science Foundation on topics ranging 
from getting started with the Internet to 
technical specifications on using TCP/IP. 

Resources available specifically on PREPnet 
fall Into three general areas: on-line databases, 
downloadable files, and library catalogs. On- 
line databases include: PENpages 
(PSUPEN.PSU.EDU). a database of agricultural 
information made available through the 
Cooperative Extension of Penn State: EBB 
(PSUVM.PSU.EDU). a database of information 
relating to Penn State. EDIN (available through 
EBB), a database of economic statistical data; 
LUNA (NS.CC.LEHIGH.EDU), an Information 
system pertaining to Lehigh; and MEDINFO 
(MED.UPENN.EDU), a medical bulletin board 
available from the University of 
Pennsylvania. Downloadable files (available 
via anonymous FTP) include: The NIH 
(National Institute of Health) Guide 
(FTP.PnT.EDU), containing information on 
scientific initiatives and administration 
regarding extramural programs; and NIAC, the 
NASA Industrial Applications Center 
(FTP.PITT.EDU). containing information 
pertaining to research centers and laboratories 
under the direction of the U.S. government. 
Library catalogs include: LIAS 
(LIAS.PSU.EDU), Penn State; LIS 
(CMULIBRARY.ANDREW.CMU.EDU), Carnegie 
Mellon; ASA (ASA.LIB.LEHIGH.EDU). Lehigh; 
PennLIN (PENNLIB.UPENN.EDU), University 
of Pennsylvania; and. PITTCAT 
(GATE.NIC.PITT.EDU), University of 
Pittsburgh. 

Internet resources are also available to 
PREPnet users. One of the more popular of 
these resources, Usenet news (which is 
available on the Internet as well as through 
other sources), is "a giant distributed bulletin 
board system" (Coursey. 1991, p. 48). Through 
gateways to CREN (Corporation for Research 
and Educational Networking), LISTSERV's are 
also available. LISTSERV's are basically 
discussion groups on specific topics. Both of 
these resources are primarily means through 
which collaboration between colleages may 
take place. While communications between 
single users is an effective research tool, the 



414 

431 



number of people reached through Usenet or a 
LISTSERV is on a much greater scale. 
Therefore, sending out a question on either of 
these systems can result in a much more rapid 
solution to a problem. 

Moving toward the NREN 

The "High Performance Computing Act of 
1991," which is currently before Congress (S. 
272), calls for the establishment of a multi- 
gigabit National Research and Education 
Network (or NREN) by 1996. Through PREPnet, 
Pennsylvania will be in a position to take 
advantage of the NREN once it becomes reality. 

Current Nationwide Networks 

A number of government agencies, including 
the Defense Advanced Research Projects 
Agency (DARPA), the Department of Energy 
(DOE), the National Aeronautics and Space 
Administration (NASA), and the National 
Science Foundation (NSF), are already running 
independent, but Interconnected, nationwide 
networks. Some of these networks relate to 
national security and have restricted access. 
However, the largest and best known of these 
networks, NSFNET, run by the National 
Science Foundation, is relatively free of these 
restrictions. 

NSFNET is basically a nationwide backbone 
network which connects other networks. 
Through NSFNET, anyone connected to 
PREPnet can communicate with anyone else 
connected to any other regional network which 
is connected to NSFNET. NSFNET even has 
over 700 connections to international regional 
networks. The over 2,300 worldwide 
interconnected regional networks are 
collectively referred to as the Internet. 

The National Research and Education Network 

The National Research and Education Network 
can probably best be thought of as NSFNET on 
a much larger scale. While NSFNET has just 
been upgraded to a T3 network with a data 
transmission rate of 45 megabits (45 million 
bits per second), the NREN is envisioned as 
having a transmission rate of at least 1 gigabit 
(1 billion bits per second). There is even some 
disagreement as to what is meant by the 
statement that the NREN will be a gigabit 
network. NSFNET is currently running at 45 
megabits meaning that a total of 45 million 
bits of information can flow from one network 
node to the next in any given second, regardless 
of the number of individual users of the 
network sending information. When 
discussing the NREN, some people envision the 
transmission rate as 1 gigabit per user, and not 



Just 1 gigabit for the entire network. The term 
terabit (1 trillion bits of information per 
second) is even beginning to be used to describe 
transmission rates. 

The NREN Is envisioned as a prototype 
network. It is expected that private enterprise 
will create similar networks to provide similar 
capabilities to the commercial sector, and even 
the private sector. Just as most homes 
currently have phone and cable television 
connections, by the turn of the century network 
connections to the home may be a reality. 
Gateways will be established to allow 
information to flow between the NREN and 
these private networks. A number of large 
corporations have already invested major 
amounts to make this vision a reality. With the 
unique blend of educational, governmental, 
and commercial entities currently connected to 
PREPnet, networking in Pennsylvania is in a 
unique position to evolve as a part of this 
nationwide information infrastructure. 
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